Real-time integration and review of dance performances streamed from remote locations

ABSTRACT

Systems and methods are provided for assembling and displaying a visual ensemble of musical performances that were created and uploaded from one or more locations that are remote from a host of the network, a director or other administrator reviewing submissions for selection and assembly, or perhaps merely remote from one or more other submissions received over a computer network. The assembled performances include a plurality of submissions, the submissions including performances created and uploaded at one or more locations remote from the location of the director for the assembly and display over the computer network. Systems and methods are also included for mapping one performance against another performance qualitatively, quantitatively, in real-time, or some combination thereof, enabling a musician, or a reviewer of performances, in the assessment of one performance relative to another performance.

BACKGROUND

This application is a continuation of U.S. application Ser. No.15/792,617, filed Oct. 24, 2017, which is a continuation of U.S.application Ser. No. 14/306,224, filed Jun. 16, 2014, now U.S. Pat. No.9,857,934, which claims the benefit of U.S. Provisional Application No.61/835,591, filed Jun. 16, 2013, each of which is hereby incorporatedherein by reference in it's entirety.

BACKGROUND Field of the Invention

The systems and methods taught herein are generally directed toassembling, displaying, and evaluating a visual ensemble of musicalperformances that include a plurality of submissions from remotelocations uploaded through a network to a director for the assembly.

Description of the Related Art

Music is a pleasure that is shared by most, and although there is a lotof musical talent to be discovered in the world, it can be a dauntingtask to discover it. Individuals that want to be discovered, forexample, can find it difficult to gain exposure, and talent scouts thatwant to find talent are limited to finding talent using currentlyavailable methods often including traveling to live performances,auditioning live, or perhaps by a manual screening of recordedsubmissions that are sent to the talent scouts from remote locations.The process of traveling, scheduling auditions, and manual reviews ofsubmissions can be expensive, as well as tedious, and quite subjective.Moreover, it can be even more challenging to assemble a band using suchtalent that has been discovered independently, from the remotelocations. Discovering talent that both looks and sounds good togetheras an ensemble of performances that work well together as a band, for atleast these reasons, requires extensive screening, travel,communications, scheduling, professional recording assistance, and thelike. This is because, currently, you can't simply receive multipleseparately recorded submissions and play them together as an ensemble insynchronization, and there's currently no technical solution to thatsynchronization problem, making live auditions necessary.

American Idol and The Voice are examples of extremely popular forums fordiscovering and displaying talent to the public, each of these forumssampling talent across the United States with the goal of discoveringnew talent for the public to enjoy. Jeff Zucker, the former NBCUniversal chief executive has been said to have described American Idolas, for example, the most impactful show in the history of television.It's not unusual for a single episode of the show to attract over 30million viewers. In its first ten years alone, American Idol createdKelly Clarkson, Carrie Underwood, Daughtry, Fantasia, Ruben Studdard,Jennifer Hudson, Clay Aiken, Adam Lambert and Jordin Sparks andremaining one of the highest rated television shows.

The process of choosing winners in such programs is long, arduous,expensive, and limiting, requiring the use of actual judges,transportation of the judges to actual physical locations for auditions,an extensive amount of time to hear the auditions, and limiting theparticipants to only those that can actual appear at the limited andactual audition sites that have been made available to the participants.The American Idol selection process, for example, includes severalrounds—an initial three rounds of preliminary additions, a Hollywoodround, a Las Vegas round (now the Sudden Death round), semi-finals, andthen finals. Although auditions can exceed 10,000 musicians in eachcity, only a few hundred make it past the preliminary round ofauditions. Those making it past the preliminary round then sing in frontof producers. If they make it past that cut, they then proceed toaudition in front of the judges, which is the only audition stage shownon television. Those selected by the judges are sent to Hollywood.Around 10-60 musicians from each city might make it to the Hollywoodround. The “top 20” musicians from the Las Vegas round move to thesemi-finals, after which the public takes over in the selection ofmusicians by voting, which has occurred through phone, text, andinternet channels. The “top 10” then move onto the finals for morevoting. The Voice, also with an incredibly high number of viewers andvoters, has a similar structure with “blind auditions”, “battle rounds”,“knockout rounds”, and live shows.

Electronic solutions to these physical limitations have been attemptedwith limited success due to the problem of having to sort through amassive number of submissions, and this, like American Idol and TheVoice, is done physically by human reviewers and, accordingly, issubjective to the reviewers. YOUTUBE, for example, is the world'slargest video-sharing website and has been credited for findingreplacements for members of popular classic rock bands like Journey,Boston and many others. While it's easy to point out a few successstories, the arduous task of sorting through the endless amount ofpotential talent is clearly overwhelming, and the answer to this problemhas not been solved, and has not been considered obvious to those ofskill.

Although these processes provide great value to the public, they stillhave several technical problems that limit their value as an effectiveauditioning tool: (i) the limitation in sampling of talent due to therequirement that auditioners and judges meet at a particular physicallocation; (ii) the limitation in obtaining the best performance by thetalent due to the stresses of performing at the particular physicallocation on demand; (iii) the limitation of having to physically reviewlarge numbers of auditions to find the best subsets of talent; (iv) thelimitation of the physical reviews being subjective to the reviewer; (v)the limitation of having to physically review a daunting number ofelectronic submissions; (vi) the inability to select different types oftalent and review them together, as an ensemble, due to the inability toalign the performances and swap-out band members at will; and, moreover,(vii) the inability to view them together using both audio and video tosee how they look, as well as perform, together as a band. Accordingly,those of skill in the art of music, auditions, and the selection oftalent will appreciate the solution to these problems set-forth herein.

SUMMARY

The systems and methods taught herein are generally directed toassembling and displaying a visual ensemble of musical performances thatinclude a plurality of submissions, each of the plurality of submissionsperformed and uploaded through a network to an administrator, such as adirector, for the assembly. The submissions include performances createdand uploaded at one or more locations remote from the location of theadministrator. Systems and methods are also included for mapping oneperformance against another performance qualitatively, quantitatively,in real-time, or some combination thereof, enabling a musician, or areviewer of performances, in the assessment of one performance relativeto another performance.

The teachings include methods of engaging in the creation of an ensembleof a set of remotely-performed and remotely-uploaded musicalperformances through a computer network having a processor operablyconnected to a memory on a non-transitory computer readable medium.Engaging in the creation of the ensemble can occur in any of a varietyof ways known to those in the music industry. For example, the engagingcan include offering, creating, hosting, sponsoring, serving as adirector in, deriving talent from, advertising on, marketing through,uploading a musical submission to, or deriving revenue from the computernetwork.

In some embodiments, the computer network includes a plurality ofmusicians in an operable communication with the computer network, theplurality of musicians having one or more musicians at a location remotefrom the location of an administrator of their performance, such as ahost, reviewer, or director, for example, that may select a set ofperformances from the plurality of musicians to create an ensemble fordisplay. The computer network also includes a timing reference on anon-transitory computer readable medium and available to the pluralityof musicians, the timing reference having a tempo for a musical workselected by the administrator and instructions for execution of thetiming reference by the processor. Moreover, the musicians willappreciate having instructions for participating in the methods, and sothe computer network can also include an instruction module on anon-transitory computer readable medium operable for providingdirections for each of the plurality of musicians. The instructionmodule can include instructions, for example, to use the timingreference to create a musical submission for the ensemble; record theirmusical submission at their remote location on a non-transitory computerreadable medium; and, upload their musical submission to the computernetwork on a non-transitory computer readable medium to enable theadministrator to select the set of performances from a plurality ofsubmissions uploaded by the plurality of musicians.

The teachings are also directed to a system for contributing to anensemble of a set of remotely-performed and remotely-uploaded musicalperformances through a computer network. In some embodiments, thesystems comprise a processor operably connected to a memory on anon-transitory computer readable medium and an input device operable toreceive audio data on a non-transitory computer readable medium. Thememory can include a database operable to store data for access on anon-transitory computer readable medium; a timing reference on acomputer readable medium, the timing reference having (i) instructionsfor execution by the processor and (ii) a tempo selected by a director,for example; an audio engine embodied in a non-transitory computerreadable storage medium, wherein the audio engine is operable totransform input audio data to output audio data; and, an instructionmodule on a non-transitory computer readable medium operable forproviding directions for a musician. For example, the instruction modulecan direct the musician to record a musical submission on anon-transitory computer readable medium at a location that is remotefrom a director of the ensemble, the recording including configuring themusical submission by following the timing reference when performing themusical submission. The instruction module can also direct the musicianto upload the musical submission to the computer network at the locationremote from the director to enable the director to create the ensembleby integrating the musical submission with the second musical submissionusing the timing interface.

It should be noted that the timing reference can offer more than a meremetronome with a constant beat. In fact, the timing reference can serveas a technical contribution to novelty and non-obviousness of theteachings set-forth herein. The timing references are configured tocreate ensembles as taught herein by functioning as a timing interface.The timing interference can serve as a technical means for allowingthose of skill to assemble the timing of the musical submission with thetiming of a second musical submission, the second musical submissionalso configured to interface with other performances in an ensemble byfollowing the timing reference when performing the second musicalsubmission. The systems, in fact, can include an integration engineembodied in a non-transitory computer readable medium and operable tointerface the timing of the musical submission with the timing of thesecond musical submission using the timing interface created by use ofthe timing reference when performing each submission assembled in theensemble.

Naturally, the methods include using a system for creating an ensembleof a set of remotely-performed and remotely-uploaded musicalperformances through a computer network. In some embodiments, thesystems can be used by any administrator of the creation of an ensemble,such as a director, a host, or a reviewer, for example. In someembodiments, the administrator can include any person desiring to selectand assemble an ensemble of performances from independent performancesuploaded by the plurality of musicians to the computer network. In someembodiments, the administrator can include one of the plurality ofmusicians. Such a system can have a processor operably connected to amemory on a non-transitory computer readable medium, the memoryincluding, for example, a receiving module operable for receiving audiodata from a musical submission and a second musical submission on anon-transitory computer readable medium; an integration engine embodiedin a non-transitory computer readable medium wherein the integrationengine is operable to interface the timing of the musical submissionwith the timing of the second musical submission using the timinginterface created by use of the timing reference in the creation of eachsubmission assembled in the ensemble. And, a system for reviewingsubmissions, and creating and reviewing an ensemble, can include adisplay module operable for displaying the ensemble.

The musicians making submissions, and the administrators receivingand/or rating the submissions, will appreciate having a tool that willhelp them compare the frequency and timing of a musical submission withthat of a reference performance. Such a tool can facilitate the learningby musicians, review of performances, and selection of musicians. Assuch, the systems and methods taught herein can further comprise aguidance module on a non-transitory computer readable medium andoperable as a guidance system that includes a timing comparison tool tocompare the musical submission to the select reference performance, theguidance system providing a feedback that includes a qualitative orquantitative measure of the timing comparison. Likewise, the guidancesystem can include a frequency comparison tool to compare the musicalsubmission to the select reference performance, the guidance systemproviding a feedback that includes a qualitative or quantitative measureof the frequency comparison. Likewise, the guidance system can includeboth a frequency comparison tool and a timing comparison tool providingat least two-dimensions to compare the musical submission to the selectreference performance, the guidance system providing a feedback thatincludes a qualitative or quantitative measure of the frequencycomparison and the timing comparison.

The guidance system can be dynamic, providing an ongoing, continuouscomparison of frequency and timing between a musical submission and it'sreference performance over the course of the performance of the musicalsubmission. In some embodiments, the dynamic guidance module can be on anon-transitory computer readable medium and operable as a dynamicguidance system including a frequency comparison tool and a timingcomparison tool to provide a continuous comparison of the musicalsubmission with the select reference performance from the beginning ofthe performance to the end of the performance, or any section therein,the dynamic guidance system providing a qualitative or quantitativemeasure of the frequency comparison and the timing comparison. In someembodiments, the dynamic guidance module can be on a non-transitorycomputer readable medium and operable as a dynamic guidance system, theguidance system including a frequency comparison tool and a timingcomparison tool for each of the plurality of musicians to continuouslycompare their performance of the musical submission with the selectreference performance while they perform, the dynamic guidance systemproviding a real-time feedback that includes a qualitative orquantitative measure of the frequency comparison and the timingcomparison, the real-time feedback being no slower than 30 millisecondsfrom the occurrence of a respective transient or note performed in themusical submission and used for comparison to the select referenceperformance.

The timing reference is robust in it's function as a timing interface inthat it accounts for tempo changes, complex rhythms and beats, and forthat matter, any variable timing that occurs throughout a complexmusical performance. In some embodiments, the timing reference caneffectively “normalize” the timing of a plurality of performances thatwere performed at locations that were remote from the location of anadministrator, host, director, or any reviewer that desires to assembleany combination of the plurality of performances as an ensemble. In someembodiments, the timing reference can be selected from the groupconsisting of a click-track; a drum track; a select reference track of aselect reference performance, J i, the select reference performancehaving a series of sound frequencies, each of the frequencies performedat a particular time in the select reference performance; a residualaudio track representing a multi-track digital audio file of a musicalwork having the select reference performance removed, the residual audiotrack having a series of sound frequencies, each of the frequenciesperformed at a particular time in the residual audio track; and, anycombination thereof.

In some embodiments, the timing reference is one of a series ofmulti-track digital audio files available upon request by each of theplurality of musicians; each of the series containing a shared musicalwork, highlighting a select reference performance on a select referencetrack within the musical work. In these embodiments, the timingreference can include T reference performances within the musical work,wherein T=Σ

; and,

_(i)=

₁,

₂,

₃, . . . , or

_(T); a select reference track of the select reference performance,

_(i), the select reference performance having a series of soundfrequencies, each of the frequencies performed at a particular time inthe select reference performance; a residual audio track representingthe multi-track digital audio file having the select referenceperformance removed; and, instructions for execution by the processor.

As such, the teachings are also directed to a method of facilitating thecreation of an ensemble of a set of remotely-performed andremotely-uploaded musical performances through a computer network. Insome embodiments, the method can include engaging in a computer networkhaving a processor operably connected to a memory on a non-transitorycomputer readable medium. The teachings include methods of engaging inthe creation of an ensemble of a set of remotely-performed andremotely-uploaded musical performances through a computer network havinga processor operably connected to a memory on a non-transitory computerreadable medium.

In some embodiments, the computer network includes a plurality ofmusicians in an operable communication with the computer network, theplurality of musicians having one or more musicians at a location remotefrom the location of a host, or other reviewer of their performance,such as a director, for example, that may select a set of performancesfrom the plurality of musicians to create an ensemble for display. And,in some embodiments, the computer network also includes a timingreference on a non-transitory computer readable medium and available tothe plurality of musicians, the timing reference having a tempo for amusical work selected by the director and instructions for execution ofthe timing reference by the processor.

Moreover, the musicians will appreciate having instructions forparticipating in the methods, and so the computer network can alsoinclude an instruction module on a non-transitory computer readablemedium operable for providing directions for each of the plurality ofmusicians. The instruction module can include instructions, for example,to use the timing reference to create a musical submission for theensemble, record their musical submission at their remote location on anon-transitory computer readable medium, and upload their musicalsubmission to the computer network on a non-transitory computer readablemedium to enable the director to select the set of performances from aplurality of submissions uploaded by the plurality of musicians.

In some embodiments, the reviewer is a director that selects a set ofperformances, Σ

_(ir)≤

_(ir),

_(2r),

_(3r), . . . , +

_(Tr), from the plurality of musicians in creating an ensemble.Moreover, the computer network can include a series of multi-trackdigital audio files available upon request by each of the plurality ofmusicians; each of the series containing a shared musical work,highlighting a select reference performance on a select reference trackwithin the musical work and including T reference performances withinthe musical work, wherein T=Σ

_(i); and,

_(i)=

₁,

₂,

₃, . . . , or

_(T); a select reference track of the select reference performance,

_(i), the select reference performance having a series of soundfrequencies, each of the frequencies performed at a particular time inthe select reference performance; a residual audio track representingthe multi-track digital audio file having the select referenceperformance removed; and, instructions for execution of each of theseries by the processor. And, as described above, the computer networkcan also provide directions, often through the use of an instructionmodule on a non-transitory computer readable medium, instructing to eachof the plurality of musicians to request one of the series ofmulti-track digital audio files having the select reference performancethat they desire to replace with their own submission,

_(ir); record their submission at their remote location on anon-transitory computer readable medium; perform their submission usingthe select reference track, the residual audio track, or the combinationthereof, as a timing interface for integration of the ensemble; and,upload their submission to the computer network on a non-transitorycomputer readable medium to enable the director to select the set ofperformances from a plurality of submissions uploaded by the pluralityof musicians.

As described above, the engaging in the creation of the ensemble canoccur in any of a variety of ways known to those in the music industry.For example, the engaging can include offering, creating, hosting,sponsoring, serving as a director in, deriving talent from, advertisingon, marketing through, uploading a musical submission to, or derivingrevenue from the computer network. In some embodiments, the engagingincludes directing, and the method further comprises reviewing theplurality of submissions, each of the plurality of submissions on anon-transitory computer readable medium; selecting the set ofperformances; and, interfacing the set of performances with anintegration engine on a computer readable medium to create the ensemblefor displaying with a display module on a non-transitory computerreadable medium.

The teachings are also directed to a method for a musician to contributeto an ensemble of musical performances from a location that is remotefrom the location of a director that is creating the ensemble over acomputer network. In some embodiments, the method includes entering acomputer network having a processor operably connected to a memory on anon-transitory computer readable medium; a plurality of musicians, eachof the plurality of musicians at a remote location from the director andin an operable communication with the computer network; a director thatselects a set of performances, Σ

_(ir)≤

_(1r),

_(2r),

_(3r), . . . , +

_(Tr), from the plurality of musicians in creating an ensemble; a seriesof multi-track digital audio files available upon request by each of theplurality of musicians; each of the series containing a shared musicalwork. Each of the series, in fact, highlight a select referenceperformance on a select reference track within the musical work, andincluding (i) T reference performances within the musical work, whereinT=Σ

_(i); and,

_(i)=

₁,

₂,

₃, . . . , or

_(T); (ii) a select reference track of the select reference performance,

_(i), the select reference performance having a series of soundfrequencies, each of the frequencies performed at a particular time inthe select reference performance; (iii) a residual audio trackrepresenting the multi-track digital audio file having the selectreference performance removed; and, (iv) instructions for execution ofeach of the series by the processor. The computer network also includesdirections, often provided on an instruction module on a non-transitorycomputer readable medium, instructing each of the plurality of musiciansto request one of the series of multi-track digital audio files havingthe select reference performance that they desire to replace with theirown musical submission,

_(ir); perform their musical submission at their remote location usingthe select reference track, the residual audio track, or the combinationthereof, as a timing interface for integration of the ensemble; recordtheir musical submission at their remote location on a non-transitorycomputer readable medium; and, upload their submission to the computernetwork on a non-transitory computer readable medium to enable thedirector to select the set of performances from a plurality ofsubmissions uploaded by the plurality of musicians.

In some embodiments, the method includes requesting from the computernetwork one of the series of multi-track digital audio files having theselect reference performance that is to be replaced with a submission,

_(ir); recording the submission on a non-transitory computer readablemedium at a location remote from the director while using the selectreference performance as a guide for the timing of performing each ofthe frequencies, and using the residual audio track as an accompaniment;and, uploading the submission to the database on a non-transitorycomputer readable medium for an interfacing of timing of the submissionwith the timing of a second submission that was recorded on anon-transitory computer readable medium by a second musician at a secondlocation remote from the director while using the select referenceperformance as the timing interface, and using the residual audio trackas an accompaniment. The methods can also include a displaying of thesubmission with the second submission as an ensemble, and the displayingcan be obtained using a display module on a non-transitory computerreadable medium operable to display each performance, and the ensembleof performances on a graphical user interface.

The systems and methods taught herein can include the assembly of bothaudio and video data in the creation of an ensemble. In this way, amusician or reviewer can visualize how a performance appears and soundsas an ensemble. Performers can be swapped-out to view a variety ofdifferent combinations in the selection of a set of performances toassemble as an ensemble, or band. As such, in some embodiments, theensemble is a visual ensemble, and each performance in the set ofperformances includes a video recording of the respective submission fordisplay in the visual ensemble. Likewise, in some embodiments, thesystems can be configured to include an input device operable to receivevideo data on a non-transitory computer readable medium; a video engineembodied in a non-transitory computer readable medium, wherein the videoengine is operable to transform input video data to output video data asa video recording of a musical submission for display in a visualensemble; and, an output module embodied in a non-transitory computerreadable medium, wherein the output module is operable to transmit theintegrated audio data, along with the video data, to an output device,the output device operable to display the visual ensemble.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 shows a general technology platform for the system, according tosome embodiments.

FIGS. 2A and 2B illustrate a processor-memory diagram to describegeneral components of the systems taught herein, according to someembodiments.

FIG. 3 is a concept diagram illustrating the system, according to someembodiments.

FIGS. 4A-4C illustrate the system, according to some embodiments: (A) alearning method; (B) a method of facilitating the creation of anensemble of a set of remotely-performed and remotely-uploaded musicalperformances; and, (C) a method for a musician to contribute to anensemble of musical performances from a location that is remote from thelocation of a director that is creating the ensemble.

FIG. 5 illustrates a circuit diagram of the system according to someembodiments.

FIG. 6 illustrates a circuit diagram of the system including a clickfunction, according to some embodiments.

FIGS. 7A and 7B illustrate a musical notation display designed for usewith the system, according to some embodiments.

FIGS. 8A-8C illustrates the system with a variety of functions for avariety of musical instruments, according to some embodiments.

FIG. 9 illustrates a view of a couple dancing in an audio/video dancingdemonstration, according to some embodiments.

FIG. 10 illustrates a display for a graphical user interface offeringselections within a track list of a multi-track audio recording setdesigned for use with the system, according to some embodiments.

FIG. 11 illustrates a display for a graphical user interface offeringinformation on the track list selection in the system, according to someembodiments.

FIG. 12 illustrates a display for a graphical user interface offeringstate selections for track faders, volume fader control, and transportselections, in which all tracks are selected on a multi-track audiorecording designed for use with the system, according to someembodiments.

FIG. 13 illustrates a display for a graphical user interface offeringstate selections for track faders, volume fader control, and transportselections, in which all tracks but the click track are selected on amulti-track audio recording designed for use with the system, accordingto some embodiments.

FIG. 14 illustrates a display for a graphical user interface offeringstate selections for track faders, volume fader control, and transportselections, in which only the isolated instrument audio track and theemulated instrument audio track are selected on a multi-track audiorecording designed for use with the system, according to someembodiments.

FIG. 15 illustrates a display for a graphical user interface offeringstate selections for track faders, volume fader control, and transportselections, in which only the residual track and the emulated instrumentaudio track are selected on a multi-track audio recording designed foruse with the system, according to some embodiments.

FIG. 16 illustrates a display for a graphical user interface offeringmusical transcription and tablature audio data, in which only thecurrent section of music is shown for a multi-track audio recordingdesigned for use with the system, according to some embodiments.

FIG. 17 illustrates a display for a graphical user interface offeringselection of a section of a piece of music by bar or set of bars in amulti-track audio recording designed for use with the system, accordingto some embodiments.

FIG. 18 illustrates a display for a graphical user interface offering ahelp page for the system, according to some embodiments.

FIG. 19 shows how a network may be used for the system, according tosome embodiments.

FIGS. 20A-20F show how an x-y guidance system can be used to help themusician stay “in the pocket” by monitoring note frequency and timingduring the course of a performance, according to some embodiments.

FIG. 21 shows how a timeline guidance system can be used to help themusician stay “in the pocket” by monitoring note frequency and timingduring the course of a performance, according to some embodiments.

FIG. 22 is and illustration of how video selections of multipleperformances can be selected and placed on a graphical user interfacefor concurrent viewing, according to some embodiments.

DETAILED DESCRIPTION OF THE INVENTIONS

The systems and methods taught herein are generally directed toassembling and displaying a visual ensemble of musical performances thatinclude a plurality of submissions, each of the plurality of submissionsperformed and uploaded through a network to an administrator, such as adirector, for the assembly. The submissions include performances createdand uploaded at one or more locations remote from the location of theadministrator. Systems and methods are also included for mapping oneperformance against another performance qualitatively, quantitatively,in real-time, or some combination thereof, enabling a musician, or areviewer of performances, in the assessment of one performance relativeto another performance.

The systems and methods can assemble an ensemble of audio only, or audioand video, using any one or any combination of the teachings providedherein. In some embodiments, the performances can include a videoportion, the systems and methods providing a variety of functionalitiesand uses. On the user side, for example, the systems and methods allowfor some control of tempo, and the tempo is mapped, sometimes fixed,sometimes variable, and sometimes a variable can be adjusted to “conformto fixed tempo”, in which the system can providing a moving average oftempo in any of a series of variable tempo sections, the sections parsedby tempo according to the musical work. Several users can perform worksindependently, and the tempo map allows them to assemble and displaytheir combined works according to a set tempo map by system default, ora master-slave relationship. This can be done publicly or in privaterooms. Likewise, on the admin side, the system allows reviewers toreview performances by comparing a user's performance to a templateperformance, by comparing one user's performance to another user'sperformance, or some combination thereof. There can also be a guidancesystem and/or rating feature that maps the user's audio portion to atemplate portion to evaluate the user's performance against thetemplate. The guidance system and/or rating feature can be a defaultprogram or custom design for particular desired performance traits whichmay define, for example, the “groove”, “feel”, or “pocket” of aparticular performance or set of performances. The custom design can bea publicly disclosed design, or it can be proprietary to the reviewers.

The systems and methods taught herein are generally directed to adynamic point referencing, or tempo mapping, of an audiovisualperformance for an accurate and precise selection and controlled cyclingof portions of the performance. In some embodiments, the teachingsherein can include the development of a tempo map, and the tempo map cancomprise a mapping of a complex tempo. In some embodiments, for example,the teachings are directed to a method of creating the instructionalwork of a performing art. Such methods can include selecting the work;and, mapping the audio portion of the work to create a dynamic pointreference. The dynamic point reference is used to facilitate an accurateand precise selection, and point-cycling, of a desired portion of thework. In some embodiments, the dynamic point reference includes a tempomap of the audio portion that is adaptable for a plurality of temposwithin the work. The term “dynamic point reference” can also be referredto as a “map,” “a tempo map,” “a customized tempo map,” “a timingreference,” “a variable timing reference,” “a manually created andaudible, variable timing reference,” “an index,” “indexing,” “a customindex,” “a customized indexing,” “grid,” “tempo/index,” “position pointreference,” “variable point reference,” or the like, in someembodiments.

The terms “audio/video,” “audiovisual,” “audio/visual,” “AV,” and thelike, can be used interchangeably in most embodiments, the termstypically being used to refer to a work having an audio track or audioportion and a corresponding video portion or video portion. The term“corresponding” can refer, for example, to (i) the original source filerecorded with the video; or, (ii) a recreated file in which the audioand/or video has been recreated or added, making the audio/video fileuseful for the teachings provided herein. Audiovideo files can include,for example, a monotrack audio file, a stereo audio file, or amulti-track audio file. A “multitrack file” can include, for example,from about 3 tracks to over 200 tracks, in some embodiments. And, itshould be appreciated that a multi-track audio file can be analog ordigital and, in fact, it can be extracted from a mono- or stereo-mixedfile, or it can be extracted from any type of audio file sharingmultiple instruments. The term “extracted” can be used to refer to aseparation of instruments, in some embodiments, that are combined in anaudio file. One of skill will readily identify such extraction methodsknown in the art including, but not limited to, processes that usephase- or phase-reversal techniques or frequency techniques (e.g.,fast-fourier transform, FFT, discrete fourier transform, DFT, etc),mathematical algorithms, or a combination thereof, for example. Suchextractions can be used with the teachings herein, in some embodiments.

The dynamic point referencing can be used by a learning artist, forexample, in analyzing or performing a portion of the work through anaccurate and precise digital audio/video instructional method having thecontrolled cycling feature. Such systems and methods will beappreciated, for example, by musicians, dancers, and other enthusiastsof the performing arts. In some embodiments, the work comprises an audioportion and a video portion, wherein the video portion includes aperformance of the work by a performing artist. The terms “artist,”“performing artist,” “performer,” “musician,” “instructional artist,”“instructor,” “teacher,” and the like, can be used interchangeably insome embodiments. In these embodiments, a dynamic point reference can beincluded within the musical work for an accurate and precise selectionand point-cycling of a desired portion of the work. The dynamic pointreference can include a tempo map of the musical work that is adaptablefor a plurality of tempos within the musical work. Examples of tempomapping can be found, for example, in U.S. Pat. No. 7,902,446, and U.S.application Ser. No. 13/274,293, each of which is hereby incorporatedherein in its entirety by reference.

In some embodiments, the teachings are directed to a method for a userto learn a performance through an accurate and precise audiovisualinstructional process. In these embodiments, the method can includeobtaining an audiovisual work having an audio portion, a video portion,and a dynamic point reference for selecting a desired portion of theaudiovisual work. The method can also include selecting the desiredportion of the work using the dynamic point reference and emulating theperformance. The method can also include point-cycling the desiredportion until the desired portion is learned to a satisfaction of theuser. The term “user” can include any operator of the system which, insome embodiments, can include a “learning artist,” “student,”“observer,” “analyst,” “voter,” “judge,” “teacher”, “instructor,”“competitor”, “competing artist”, “competing performer,” and the like.

An advantage of the teachings provided herein is the accuracy andprecision available in the selection of a desired portion. Due to thisadvantage, the cycling can include any selected time-frame within thework, such as a period of silence in an audio track, or any one or anyseries of sounds, beats, and bars. In some embodiments, the cycling canconsist of a single musical note, a single repeating rhythmic pattern,or a series of accurate and precise selections from a tempo map. Eachselection in the series of selections can consist of silence, a musicalnote, a plurality of beats, a bar, a plurality of bars, a repeatingrhythmic pattern, or any combination thereof. In some embodiments, avisual portion of the work can provide a point of reference for thecycling such as, for example, during a period of silence in the work, orwhere an audio portion of the work is otherwise insufficient toestablish a point of reference for the cycling, such as where the audioportion is damaged or ineffective. The ability to accurately andprecisely select and cycle a desired portion of a musical work is avaluable aspect of the teachings provided herein. The terms “cycling,”“point cycling,” “rhythmic cycling,” “looping,” and “controlledcycling,” can refer to the repeated playback of a desired portionselected using the teachings provided herein.

The terms “accurate,” “accurately,” and the like, can be used to referto a measurable error, or in some embodiments the absence of ameasurable error, in the difference between a user's actual desiredportion, and the user's ability to select the desired portion from awork using the teachings provided herein. And, the desired portion canbe a portion of an audio track or a related video portion. The terms“precise,” “precisely,” “precision,” and the like can be used to referthe user's ability to reproducibly select the desired portion. The highdegree of accuracy provided through the systems and methods taughtherein produces, in response, likewise results in a high degree ofprecision for the user as well.

In some embodiments, the systems methods provided herein results insubstantially no error in the selecting of the desired portion, suchthat any error may be isolated to fall within a relatively high accuracyof a computing system performing the functions taught herein which, insome embodiments, any error in selecting the desired portion can be lessthan a fraction of a millisecond. Such a computing system error may be,for example, less than about 3 milliseconds in some embodiments, betweenabout 0.5 milliseconds and about 3 milliseconds in other embodiments,and less than perhaps a single millisecond in yet other embodiments. Theterm “desired portion” can be used to refer to a portion of a musicalwork that may range in duration, for example, from about 50 millisecondsto about 20 minutes. In some embodiments, a user can try to select adesired portion having a time duration of play that is very short, forexample, ranging from about 90 ms to perhaps about 0.5 seconds or 1second, making it difficult for the user to select the desired portionin a real-time manner. In this instance, such user error resulting fromuser response time is mitigated by offering the various features taughtherein that include, for example, (i) a “nudge” function that allows theuser to move forward in a preselected number of beats or any timereference, such as a single beat, fraction of a beat, pairs of beats,sets of beats, bars, pairs of bars, sets of bars, and the like; (ii)rules that automatically provide a region around an area selected, suchthat the user can be too slow or too fast in response time and stillcapture the desired portion; and (iii) a tempo adjustment to allow theuser to, for example, slow down a whole work, or particular section tofacilitate ease of selection of the desired portion.

FIG. 1 shows a general technology platform for the system, according tosome embodiments. The computer system 100 may be a conventional computersystem and includes a computer 105, I/O devices 150, and a displaydevice 155. The computer 105 can include a processor 120, acommunications interface 125, memory 130, display controller 135,non-volatile storage 140, and I/O controller 145. The computer system100 may be coupled to or include the I/O devices 150 and display device155.

The computer 105 interfaces to external systems through thecommunications interface 125, which may include a modem or networkinterface. It will be appreciated that the communications interface 125can be considered to be part of the computer system 100 or a part of thecomputer 105. The communications interface 125 can be an analog modem,isdn modem, cable modem, token ring interface, satellite transmissioninterface (e.g. “direct PC”), or other interfaces for coupling thecomputer system 100 to other computer systems. In a cellular telephone,this interface is typically a radio interface for communication with acellular network and may also include some form of cabled interface foruse with an immediately available personal computer. In a two-way pager,the communications interface 125 is typically a radio interface forcommunication with a data transmission network but may similarly includea cabled or cradled interface as well. In a personal digital assistant,the communications interface 125 typically includes a cradled or cabledinterface and may also include some form of radio interface, such as aBLUETOOTH or 802.11 interface, or a cellular radio interface, forexample.

The processor 120 may be, for example, any suitable processor, such as aconventional microprocessor including, but not limited to, an IntelPentium microprocessor or Motorola power PC microprocessor, a TexasInstruments digital signal processor, or a combination of suchcomponents. The memory 130 is coupled to the processor 120 by a bus. Thememory 130 can be dynamic random access memory (DRAM) and can alsoinclude static ram (SRAM). The bus couples the processor 120 to thememory 130, also to the non-volatile storage 140, to the displaycontroller 135, and to the I/O controller 145.

The I/O devices 150 can include a keyboard, disk drives, printers, ascanner, and other input and output devices, including a mouse or otherpointing device. The display controller 135 may control in theconventional manner a display on the display device 155, which can be,for example, a cathode ray tube (CRT) or liquid crystal display (LCD).The display controller 135 and the I/O controller 145 can be implementedwith conventional well known technology, meaning that they may beintegrated together, for example.

The non-volatile storage 140 is often a FLASH memory or read-onlymemory, or some combination of the two. A magnetic hard disk, an opticaldisk, or another form of storage for large amounts of data may also beused in some embodiments, although the form factors for such devicestypically preclude installation as a permanent component in somedevices. Rather, a mass storage device on another computer is typicallyused in conjunction with the more limited storage of some devices. Someof this data is often written, by a direct memory access process, intomemory 130 during execution of software in the computer 105. One ofskill in the art will immediately recognize that the terms“machine-readable medium” or “computer-readable medium” includes anytype of storage device that is accessible by the processor 120 and alsoencompasses a carrier wave that encodes a data signal. Objects, methods,inline caches, cache states and other object-oriented components may bestored in the non-volatile storage 140, or written into memory 130during execution of, for example, an object-oriented software program.

The computer system 100 is one example of many possible differentarchitectures. For example, personal computers based on an Intelmicroprocessor often have multiple buses, one of which can be an I/O busfor the peripherals and one that directly connects the processor 120 andthe memory 130 (often referred to as a memory bus). The buses areconnected together through bridge components that perform any necessarytranslation due to differing bus protocols.

In addition, the computer system 100 can be controlled by operatingsystem software which includes a file management system, such as a diskoperating system, which is part of the operating system software. Oneexample of an operating system software with its associated filemanagement system software is the family of operating systems known asWindows CEO and Windows® from Microsoft Corporation of Redmond, Wash.,and their associated file management systems. Another example ofoperating system software with its associated file management systemsoftware is the LINUX operating system and its associated filemanagement system. Another example of an operating system software withits associated file management system software is the PALM operatingsystem and its associated file management system. Another example of anoperating system is an ANDROID, or perhaps an iOS, operating system. Thefile management system is typically stored in the non-volatile storage140 and causes the processor 120 to execute the various acts required bythe operating system to input and output data and to store data inmemory, including storing files on the non-volatile storage 140. Otheroperating systems may be provided by makers of devices, and thoseoperating systems typically will have device-specific features which arenot part of similar operating systems on similar devices. Similarly,WinCEO, PALM, IOS or ANDROID operating systems, for example, may beadapted to specific devices for specific device capabilities.

The computer system 100 may be integrated onto a single chip or set ofchips in some embodiments, and can be fitted into a small form factorfor use as a personal device. Thus, it is not uncommon for a processor,bus, onboard memory, and display/I-O controllers to all be integratedonto a single chip. Alternatively, functions may be split into severalchips with point-to-point interconnection, causing the bus to belogically apparent but not physically obvious from inspection of eitherthe actual device or related schematics.

FIGS. 2A and 2B illustrate processor-memory diagrams to describecomponents of the system, according to some embodiments. In FIG. 2A, thesystem 200 shown in FIG. 2 contains a processor 205 and a memory 210(that can include non-volatile memory), wherein the memory 210 includesan audio/video database 215, a transformation module 220, an emulationrecording module 225, an integration engine 230, an output module 235,and an optional video display module 240, which can also be a part ofthe output module 235. The system can further comprise an optional dataexchange module 245 embodied in a non-transitory computer readablemedium, wherein the data exchange module is operable to exchange datawith external computer readable media.

The system includes an input device (not shown) operable to receiveaudio data or video data on a non-transitory computer readable medium.Examples of input devices include a data exchange module operable tointeract with external data formats, voice-recognition software, ahand-held device in communication with the system including, but notlimited to, a microphone, and the like, as well as a camera or othervideo image capture and transmission device. It should be appreciatedthat the input and output can be an analog or digital audio or video.

The audio/video database 215 is operable to store audio or video filesfor access on a non-transitory computer readable storage medium. In someembodiments, the system can store original multi-track audio files,copies of original multi-track audio files, and the like. Any audio orvideo file known to one of skill in the art can be stored including, butnot limited to sound files, text files, image files, and the like. Insome embodiments, the system can access any of a variety of accessibledata through a data exchange module, as discussed above.

Any audio or video format known to one of skill in the art can be used.In some embodiments, the audio file comprises a format that supports oneaudio codec and, in some embodiments, the audio file comprises a formatthat supports multiple codecs. In some embodiments the audio filecomprises an uncompressed audio format such as, for example, WAV, AIFF,and AU. In some embodiments, the audio file format comprises losslesscompression such as, FLAC, Monkey's Audio having file extension APE,WavPack having file extension WV, Shorten, Tom's lossless AudioKompressor (TAK), TTA, ATRAC Advanced Lossless, Apple Lossless, andlossless WINDOWS Media Audio (WMA). In some embodiments, the audio fileformat comprises lossy compression, such as MP3, Vorbis, Musepack,ATRAC, lossy WINDOWS Media Audio (WMA) and AAC.

In some embodiments, the audio format is an uncompressed PCM audioformat, as a “.wav” for a WINDOWS computer readable media, or as a“.aiff” as a MAC OS computer readable media. In some embodiments aBroadcast Wave Format (BWF) can be used, allowing metadata to be storedin the file. In some embodiments, the audio format is a lossless audioformat, such as FLAC, WavPack, Monkey's Audio, ALAC/Apple Lossless. Insome embodiments, the lossless audio format provides a compression ratioof about 2:1. In some embodiments, the audio format is a free-and-openformat, such as wav, ogg, mpc, flac, aiff, raw, au, or mid, for example.In some embodiments, the audio format is an open file format, such asgsm, dct, vox, aac, mp4/m4a, or mmf. In some embodiments the audioformat is a proprietary format, such as mp3, wma, atrac, ra, ram, dss,msv, dvg, IVS, m4p, iklax, mxp4, and the like.

The transformation module 220 is operable to transform a multi-trackaudio file comprising an isolated instrument audio track and a residualcomponent track into a ratio of (i) the isolated instrument audio trackto (ii) the residual component track, wherein the residual componenttrack represents a subtraction of the isolated instrument audio trackfrom the plurality of audio tracks, and the transforming can resultsfrom a user selecting a gain ratio, for example, between the isolatedinstrument audio track, the residual component track, and a click track.It should be appreciated that a “gain ratio” can be used to refer to auser-controlled variable sound level relationship between the minimum(inaudible) sound volume (infinity:1) to maximum loudness output (0 dBfull scale with a ratio of 1:1). The terms “gain” and “volume” can beused interchangeably in some embodiments, where a gain of “0” can beused, in some embodiments, as a reference for a minimum volume of anaudio portion, track or otherwise; and, a ratio of 0 can be used torefer to a gain in the numerator of 0. For example, a ratio of anisolated instrument audio track to a residual component audio track of 0can mean, for example, that at least the isolated instrument audio trackhas been turned off, or at least down to the minimum volume setting of0. This setting allows the residual component volume, or gain, to beadjusted to an audible level desired by a user. Likewise, a ratio of aresidual component audio track to an isolated instrument audio track of0, for example, can mean that the residual component audio track hasbeen turned off, or at least down to the minimum volume setting of 0,such that the isolated instrument audio track can be adjusted to adesired audible level for play in the absence of the residual componentaudio track.

The “residual component track” or “residual audio track” can be referredto as “a background audio track” or “an emulation audio track,” in someembodiments. The term “metronome track” can be referred to as a “clicktrack,” “manually created, audible, variable timing reference track,”“audible, variable timing reference track,” “variable timing referencetrack,” “audible, dynamic point reference track,” “audible, positionpoint reference track,” “audible, variable point reference track,” andthe like, in some embodiments. It should be appreciated that, in someembodiments, the metronome track can provide a steady tempo. However,the term “metronome track” can also be used, in some embodiments, torefer to a track having a variable tempo for use with the dynamic pointreferencing taught herein.

The residual track represents a subtraction of the isolated instrumentaudio track from the plurality of audio tracks, and the transforming canresult from a user selecting a between the isolated instrument audiotrack, the residual track, and the click track. The system can alsoinclude an emulation recording module 225 embodied in a non-transitorycomputer readable medium. The emulation recording module 225 is operableto record the user's emulated audio track on a non-transitory computerreadable medium. In some embodiments, the emulation recording module 225can be operable within a single functioning section of a system, such asa single page of a software application. In some embodiments, theemulation recording module 225 can be operable within a plurality offunctioning sections of a system, such as in a plurality of pages of asoftware application, such that the recording can occur quickly at thechoosing of the user without having to move from one section of thesystem to another section of the system.

The system can also include an integration engine 230 embodied in anon-transitory computer readable medium, wherein the integration engine230 is operable to combine the emulated instrument audio track with theresidual track to transform the multi-track audio file into aneducational audio file. In addition, the system can include an outputmodule 235 embodied in a non-transitory computer readable medium,wherein the output module 235 is operable to transmit audio data to anoutput device, which can be a graphical user interface, or videodisplay, which can optionally be supported by a separate video displaymodule 240, or the display can be supported with one or more otheroutput devices by the output module 235. The output device can beoperable to provide audio data to the user, wherein the audio dataassists the user in learning a preselected piece of music.

In some embodiments, the input device comprises a microphone and/orcamera and, in some embodiments, the output module 235 transmits musictranscription and tablature data, or an alternative visualrepresentation that may or may not be traditional transcription andtablature data, such as an animated motion graphic or cartoon, of aninstrument or performance, to a graphical user interface. In someembodiments, the output device comprises a speaker, a graphical userinterface, or both a speaker and a graphical user interface, forexample. And, in some embodiments, the output module has asynchronization function operable to synchronize the music transcriptionand tablature data display on the graphical user interface with theisolated instrument audio track provided to the listener through thespeaker.

The output module 235 can also have a recalibration function operable torecalibrate an audio data track output to correct a latency in theoutput of the audio track data. One of skill will appreciate that timestamps can be used to align, recalibrate, and correct latencies in theoutput of a data stream flow. In some embodiments, the latency iscorrected by time stamping samples of the audio data, where a “sample”is a short wave form of the audio having a length measured by anincrement of time. In some embodiments, a sample is less than a secondlong, (e.g., about 1/100 of a second, or 1/1000 of a second, induration). In some embodiments, the samples can be about 44/1000 of asecond long. An audio track can contain, for example, about 44,000samples per track per second in a high quality sound file. As such, theconcept of a sample and sample resolution is a measure of audioresolution or quality. A lower quality mono audio file can have about22,000 samples per track per second, for example.

Recalibration techniques can be used in some embodiments. Bandwidthlimitations of a computer system, for example, can create variations ortransients in misalignments between tracks and hinder sound quality. Acomputer having a smaller CPU than another computer having a larger CPUbut similar memory capacity can have latency problems that result inperformance problems. In some embodiments, the system can provide anoutput of 4 streaming files, and these files can include (i) a residualcomponent track that does not contain the isolated instrument audiotrack, (ii) an isolated instrument audio track, (iii) an emulatedinstrument audio track, and (iv) a dynamic point reference that canprovide a dynamic point referencing of a variable tempo. In someembodiments, the residual component track, the isolated instrument audiotrack, and the emulated instrument audio track are stereo files and, insome embodiments, the index track is a mono file. Each track in eachfile has it's own timeline, and there can be acceptable variances thatare not noticeable to the ear, but there can also be unacceptablevariances that result in an unacceptable and audible degradation inaudio quality.

Each input file is comprised of an array of samples, and each sample canbe used as a marker in time, since each sample position has an actualposition that will serve as a measure of variance against an idealposition. The recalibration is performed on sets of samples. When agroup of samples is off in time, the system can be designed to correctfor the next set of samples. For example, a system can be designed torecalibrate based on a sample resolution that provides a 0.001 secondaccuracy by measuring the variance of a time stamp on a group of 44samples (0.001 of a second for a 44,000 sample/sec high quality sample)to an ideal time stamp for that sample set. A fast method ofrecalibration was developed to reduce this variance or “latency” in theaudio so that it's not detectable to human being. A good ear, forexample, can hear time variance between tracks of about 1/60 of asecond, and multiple events of variances in a closely related string ofsamples can be cumulative, making it necessary to have a variance atleast an order of magnitude under 1/60 of a second. In some embodiments,the minimum audio resolution in a sample for latency correction shouldbe no less than 300 samples in a segment. In some embodiments, the audioresolution in a sample for latency correction is about 44 samples in asegment. In some embodiments, it was found that the recalibration shouldbe done on each “run through” of the data in the system, where a “runthrough” is an emptying and a filling of a data queue in the system.Between data loads in a queue, the recalibration occurs by measuring thedifference between the actual time of each track and the ideal time ofeach track and a correction is applied between data loads. In someembodiments, the audio queues up several hundred times per second.

The CPU on a handheld computer system can have difficulties concurrentlyprocessing the audio data files described herein. In some embodiments, ahandheld computing system may have latency difficulties whenconcurrently processing more than 2 audio data files. As such, datafiles may require compression. In some embodiments, the data files canbe compressed using a compression technique, for example, such asQUICKTIME by Apple. Other file compression techniques can be used. IMA4can also be used to compress the files in some embodiments. In someembodiments, the system requires at least a 600-700 MHz processor. TheiPhone has a 400 MHz processor, on the other hand, suggesting thatcompressed audio data files may be needed for use of some embodiments ofthe system on the iPhone. The IMA4 compression method compresses theaudio data file to about 25% of file size. An iPAD system can also beused in some embodiments.

In some embodiments, it should be appreciated, however, that the systemcan use pure, uncompressed wave files. Many home PCs, however, may notneed compressed files due to the more powerful processors currentlyavailable for home PCs. The bandwidth of the computer system, i.e. thesize of the CPU and memory will dictate whether compression isnecessary. One of skill in the art will appreciate that certaincompression technologies may be needed in some systems for optimumperformance and that these technologies are readily identifiable andaccessible.

One of skill will appreciate that time stamping of data samples can alsobe used to synchronize between other data streams. In some embodiments,an additional audio data stream is used to provide a digital musicaltranscription and tablature display in the form of a graphical display.This audio data can be synchronized and recalibrated at the same time asthe other audio data.

An audio file, such as a multi-track audio file, can further comprise anindex track. As such, in some embodiments, the transformation module 220can be operable to transform a multi-track audio file into a ratio of(i) the isolated instrument audio track, (ii) the residual track, and(iii) the index track, and a between the isolated instrument audiotrack, the residual track, and the index track can be selected by theuser.

As described above, the system can further comprise an optional dataexchange module 245 embodied in a non-transitory computer readablemedium, wherein the data exchange module is operable to exchange datawith external computer readable media. The data exchange module can, forexample, serve as a messaging module operable to allow users tocommunicate with other users having like subject-profiles, or othersusers in a profile independent manner, merely upon election of the user.The users can email one another, post blogs, or have instant messagingcapability for real-time communications. In some embodiments, the usershave video and audio capability in the communications, wherein thesystem implements data streaming methods known to those of skill in theart. In some embodiments, the system is contained in a hand-held device;operable to function as a particular machine or apparatus having theadditional function of telecommunications, word processing, or gaming;or operable to function as a particular machine or apparatus not havingother substantial functions.

In FIG. 2B, a flowchart shows a variation of the system 200 whereprocessor 205 is in operable connection with the memory 210 and theaudio/video database are in digital media file storage 210,215. Thesystem 200 can also include an audio engine 250 embodied in anon-transitory computer readable storage medium, wherein the audioengine 250 is operable to transform input audio data to output audiodata. Likewise, the system 200 can also include a video engine 255embodied in a non-transitory computer readable storage medium, whereinthe video engine 255 is operable to transform input video data to outputvideo data. Moreover, the system 200 can include a dynamic pointreference module 260 embodied in a non-transitory computer readablestorage medium, wherein the dynamic point reference module 260 isoperable to create a dynamic point reference for a performance using atempo map, the performance having an audiovisual file with the audiodata and the video data. The system 200 can also have an output module(not shown) embodied in a non-transitory computer readable medium,wherein the output module is operable to (i) transmit the audio data andthe video data to an output device in the form of an accurate andprecise selection of a desired portion of the performance and (ii)transmit a point-cycling of the desired portion of the performance to auser. The output device 275 is operable to provide the audio data andthe video data to the user, assisting the user in learning aperformance. The system 200 can also include a notation display engine265 to display music transcription and tablature indexed to the dynamicpoint reference. Moreover, the system 200 can include a user controlinterface 270.

The systems taught herein can be practiced with a variety of systemconfigurations, including personal computers, multiprocessor systems,microprocessor-based or programmable consumer electronics, network PCs,minicomputers, mainframe computers, and the like. The teachings providedherein can also be practiced in distributed computing environments wheretasks are performed by remote processing devices that are linked througha communications network. As such, in some embodiments, the systemfurther comprises an external computer connection through the dataexchange module 245 and a browser program module (not shown). Thebrowser program module (not shown) can be operable to access externaldata as a part of the data exchange module 245.

FIG. 3 is a concept diagram illustrating the system, according to someembodiments. The system 300 contains components that can be used in atypical embodiment. In addition to the audio database 215, thetransformation module 220, the emulation recording module 225, theintegration engine 230, and the output module 235 shown in FIG. 2, thememory 210 of the device 300 also includes a data exchange module 245and the browser program module (not shown) for accessing the externaldata. The system includes a speaker 352, display 353, and a printer 354connected directly or through I/O device 350, which is connected to I/Obackplane 340.

The system 300 can be implemented in a stand-alone device, rather than acomputer system or network. In FIG. 3, for example, the I/O device 350connects to the speaker (spkr) 352, display 353, and microphone (mic)354, but could also be coupled to other features. Such a device can havea music state selector 341, an isolated instrument audio track stateselector 342, a residual track state selector 343, a user's emulatedaudio track state selector 344, a learning state selector 345 for theeducational audio track, a bar state selector 346, and a timer stateselector 347 for the index track, with each state selector connecteddirectly to the I/O backplane 340.

In some embodiments, the system further comprises security measures toprotect the subject's privacy, integrity of data, or both. Such securitymeasures are those well-known in the art such as firewalls, software,and the like. In addition, the system can be configured for use in anenvironment that requires administrative procedures and control. Forexample, the system can include an administrative module (not shown)operable to control access, configure the engines, monitor results,perform quality assurance tests, and define audiences for targeting andtrending. Since the system can safely be provided by a network and, insome embodiments, the system is coupled to a network, the securitymeasures can help protect the contents of the system from externalintrusions.

In some embodiments, the system is a web enabled application and canuse, for example, Hypertext Transfer Protocol (HTTP) and HypertextTransfer Protocol over Secure Socket Layer (HTTPS). These protocolsprovide a rich experience for the end user by utilizing web 2.0technologies, such as AJAX, Macromedia Flash, etc. In some embodiments,the system is compatible with Internet Browsers, such as InternetExplorer, Mozilla Firefox, Opera, Safari, etc. In some embodiments, thesystem is compatible with mobile devices having full HTTP/HTTPS support,such as iPhone, PocketPCs, Microsoft Surface, Video Gaming Consoles, andthe like. In some embodiments, the system can be accessed using aWireless Application Protocol (WAP). This protocol will serve the nonHTTP enabled mobile devices, such as Cell Phones, BlackBerries, Droids,etc., and provides a simple interface. Due to protocol limitations, theFlash animations are disabled and replaced with Text/Graphic menus. Insome embodiments, the system can be accessed using a Simple ObjectAccess Protocol (SOAP) and Extensible Markup Language (XML). By exposingthe data via SOAP and XML, the system provides flexibility for thirdparty and customized applications to query and interact with thesystem's core databases. For example, custom applications could bedeveloped to run natively on iPhones, Java or .Net-enabled platforms,etc. One of skill will appreciate that the system is not limited to anyof the platforms discussed above and will be amenable to new platformsas they develop.

In some embodiments, the teachings are directed to a method for a userto learn a performance through an accurate and precise audiovisualinstructional process. In these embodiments, the method can includeobtaining an audiovisual work having an audio portion, a video portion,and a dynamic point reference for selecting a desired portion of theaudiovisual work. The method can also include selecting the desiredportion of the work using the dynamic point reference and emulating theperformance. The method can also include point-cycling the desiredportion until the desired portion is learned to a satisfaction of theuser.

The teachings include methods of engaging in the creation of an ensembleof a set of remotely-performed and remotely-uploaded musicalperformances through a computer network having a processor operablyconnected to a memory on a non-transitory computer readable medium. themethod of engaging in the creation of an ensemble of a set ofremotely-performed and remotely-uploaded musical performances includes aplurality of musicians includes engaging in a computer network. Theplurality of musicians having one or more musicians at a location remotefrom the location of an administrator of their performance, such as ahost, reviewer, or director, for example, that may select a set ofperformances from the plurality of musicians to create an ensemble fordisplay. The computer network also includes a timing reference on anon-transitory computer readable medium and available to the pluralityof musicians, the timing reference having a tempo for a musical workselected by the administrator and instructions for execution of thetiming reference by the processor. Moreover, the musicians willappreciate having instructions for participating in the methods, and sothe computer network can also include an instruction module on anon-transitory computer readable medium operable for providingdirections for each of the plurality of musicians. The instructionmodule can include instructions, for example, to use the timingreference to create a musical submission for the ensemble; record theirmusical submission at their remote location on a non-transitory computerreadable medium; and, upload their musical submission to the computernetwork on a non-transitory computer readable medium to enable theadministrator to select the set of performances from a plurality ofsubmissions uploaded by the plurality of musicians.

Engaging in the creation of the ensemble can occur in any of a varietyof ways known to those in the music industry. For example, the engagingcan include offering, creating, hosting, sponsoring, serving as adirector in, deriving talent from, advertising on, marketing through,uploading a musical submission to, or deriving revenue from the computernetwork.

FIGS. 4A-4C illustrate the system, according to some embodiments: (A) alearning method; (B) a method of facilitating the creation of anensemble of a set of remotely-performed and remotely-uploaded musicalperformances; and, (C) a method for a musician to contribute to anensemble of musical performances from a location that is remote from thelocation of a director that is creating the ensemble.

As shown in FIG. 4A, the methods 400 include obtaining 405 a multi-trackaudio file produced from a multi-track audio recording. The multi-trackaudio file can be presented with a related video portion according tothe teachings provided herein. The multi-track audio file can have anisolated instrument audio track and a residual track, and the isolatedinstrument audio track can have a single musical instrument playing apreselected piece of music that a user desires to learn on a preselectedmusical instrument. The method includes transforming 410 the compositionof a multi-track audio file to include a ratio of (i) the isolatedinstrument audio track to (ii) a residual track. The residual componentaudio track represents a subtraction of the isolated instrument audiotrack from the plurality of audio tracks. The method includes emulating415 the preselected piece of music by listening to the isolatedinstrument audio track, watching the corresponding video portion, andplaying the preselected musical instrument to create an emulatedinstrument audio track. Consistent with the teachings provided herein,the emulating can include the video instructional which can becontrolled and indexed through the tempo map as taught herein for use inconjunction with an audio track. The emulated instrument audio track isrecorded 420 on a non-transitory computer readable medium and combined425 with the residual track to transform the custom digital audio fileinto an educational audio file. The method includes listening 430 to theeducational audio track to identify deficiencies in the emulating by theuser. The user repeats 435 the emulating, recording, combining, andlistening until the user has learned the preselected piece of music onthe preselected musical instrument to the user's satisfaction.

In some embodiments, the transforming 410 includes reducing the volumeof the residual track and, in some embodiments, the transforming 410includes reducing the volume of the isolated instrument audio track. Insome embodiments, the video portion is controlled through the tempo mapas taught herein for a user to better understand how to play, andemulate the subtleties in an audio track. The method can furthercomprise selecting one or more bars of the isolated instrument audiotrack to enable the user to focus on emulating a section of thepreselected piece of music. In some embodiments, the method can includethe selection one or more bars to provide a repeated playback of thesection. In some embodiments, the emulating 415 can further comprisereading a digital musical transcription and tablature displaycorresponding to the isolated instrument audio track. And, in someembodiments, the custom digital audio file further comprises an isolatedindex track, and the method further comprises listening to the isolatedindex track that is designed for the preselected piece of music.

As shown in FIG. 4B, the methods 400 include facilitating 440 thecreation of an ensemble of a set of remotely-performed andremotely-uploaded musical performances. These methods include providing441 a system for a plurality of musicians to contribute a performance toan ensemble of a set of remotely-performed and remotely-uploaded musicalperformances through a computer network. In some embodiments, thesesystems comprise a processor operably connected to a memory on anon-transitory computer readable medium and an input device operable toreceive 445 audio data on a non-transitory computer readable medium froma musician. The memory can include a database operable to store 447 datafor access on a non-transitory computer readable medium; a timingreference on a computer readable medium for the musician to follow, thetiming reference having (i) instructions for execution by the processorand (ii) a tempo, constant, variable, or a variable set of tempos,selected by a director, for example; an audio engine embodied in anon-transitory computer readable storage medium, wherein the audioengine is operable to transform 449 input audio data to output audiodata; and, an instruction module on a non-transitory computer readablemedium operable for providing 451 directions for musicians to follow.For example, the instruction module can direct the musician to record453 a musical submission on a non-transitory computer readable medium ata location that is remote from a director of the ensemble, the recordingincluding configuring the musical submission by following 455 the timingreference when performing the musical submission. The instruction modulecan also direct the musician to upload 457 the musical submission to thecomputer network at the location remote from the director to enable thedirector to create the ensemble by integrating 459 the musicalsubmission with the second musical submission using the timinginterface.

As such, the methods can include engaging in a computer network having aprocessor operably connected to a memory on a non-transitory computerreadable medium. The teachings include methods of engaging in thecreation of an ensemble of a set of remotely-performed andremotely-uploaded musical performances through a computer network havinga processor operably connected to a memory on a non-transitory computerreadable medium. Consistent with the teachings herein, the engaging canoccur in any of a variety of ways known to those in the music industry.For example, the engaging can include offering, creating, hosting,sponsoring, serving as a director in, deriving talent from, advertisingon, marketing through, uploading a musical submission to, or derivingrevenue from the computer network. In some embodiments, the engagingincludes directing, and the method further comprises reviewing theplurality of submissions, each of the plurality of submissions on anon-transitory computer readable medium; selecting the set ofperformances; and, interfacing the set of performances with anintegration engine on a computer readable medium to create the ensemblefor displaying with a display module on a non-transitory computerreadable medium.

In some embodiments, the computer network includes a plurality ofmusicians in an operable communication with the computer network, theplurality of musicians having one or more musicians at a location remotefrom the location of a host, or other reviewer of their performance,such as a director, for example, that may select a set of performancesfrom the plurality of musicians to create an ensemble for display. And,in some embodiments, the computer network also includes a timingreference on a non-transitory computer readable medium and available tothe plurality of musicians, the timing reference having a tempo for amusical work selected by the director and instructions for execution ofthe timing reference by the processor.

Moreover, the musicians will appreciate having instructions forparticipating in the methods, and so the computer network can alsoinclude an instruction module on a non-transitory computer readablemedium operable for providing directions for each of the plurality ofmusicians. The instruction module can include instructions, for example,to use the timing reference to create a musical submission for theensemble, record their musical submission at their remote location on anon-transitory computer readable medium, and upload their musicalsubmission to the computer network on a non-transitory computer readablemedium to enable the director to select the set of performances from aplurality of submissions uploaded by the plurality of musicians.

In some embodiments, the reviewer is a director that selects a set ofperformances, Σ

_(ir)≤

_(1r),

_(2r),

_(3r), . . . , +

_(Tr), from the plurality of musicians in creating an ensemble.Moreover, the computer network can include a series of multi-trackdigital audio files available upon request by each of the plurality ofmusicians; each of the series containing a shared musical work,highlighting a select reference performance on a select reference trackwithin the musical work and including T reference performances withinthe musical work, wherein T=Σ

_(i); and,

_(i)=

₁,

₂,

₃, . . . , or

_(T); a select reference track of the select reference performance,

i, the select reference performance having a series of soundfrequencies, each of the frequencies performed at a particular time inthe select reference performance; a residual audio track representingthe multi-track digital audio file having the select referenceperformance removed; and, instructions for execution of each of theseries by the processor. And, as described above, the computer networkcan also provide directions, often through the use of an instructionmodule on a non-transitory computer readable medium, instructing to eachof the plurality of musicians to request one of the series ofmulti-track digital audio files having the select reference performancethat they desire to replace with their own submission,

_(ir); record their submission at their remote location on anon-transitory computer readable medium; perform their submission usingthe select reference track, the residual audio track, or the combinationthereof, as a timing interface for integration of the ensemble; and,upload their submission to the computer network on a non-transitorycomputer readable medium to enable the director to select the set ofperformances from a plurality of submissions uploaded by the pluralityof musicians.

The systems and methods taught herein include facilitating the creationof an ensemble by providing a musician with tools for contributing to anensemble of musical performances. The musician can contribute from alocation that is remote from the location of a director that is creatingthe ensemble. In some embodiments, the systems and methods taught hereincan be implemented by any administrator of the creation of an ensemble,such as a director, a host, or a reviewer, for example. In someembodiments, the administrator can include any person desiring to selectand assemble an ensemble of performances from independent performancesuploaded by a plurality of musicians in the computer network. In someembodiments, the administrator can include one of the plurality ofmusicians. Such a system can have a processor operably connected to amemory on a non-transitory computer readable medium, the memoryincluding, for example, a receiving module operable for receiving audiodata from a musical submission and a second musical submission on anon-transitory computer readable medium; an integration engine embodiedin a non-transitory computer readable medium wherein the integrationengine is operable to interface the timing of the musical submissionwith the timing of the second musical submission using the timinginterface created by use of the timing reference in the creation of eachsubmission assembled in the ensemble. And, a system for reviewingsubmissions, and creating and reviewing an ensemble, can include adisplay module operable for displaying the ensemble.

It should be noted that the timing reference can offer more than a meremetronome with a constant beat. In fact, the timing reference can serveas a technical contribution to novelty and non-obviousness of theteachings set-forth herein. The timing references are configured tocreate ensembles as taught herein by functioning as a timing interface.The timing interference can serve as a technical means for allowingthose of skill to assemble the timing of the musical submission with thetiming of a second musical submission, the second musical submissionalso configured to interface with other performances in an ensemble byfollowing the timing reference when performing the second musicalsubmission. The systems, in fact, can include an integration engineembodied in a non-transitory computer readable medium and operable tointerface the timing of the musical submission with the timing of thesecond musical submission using the timing interface created by use ofthe timing reference when performing each submission assembled in theensemble.

The Guidance Module

The musicians making submissions, and the administrators receivingand/or rating the submissions, will appreciate having a tool that willhelp them compare the frequency and timing of a musical submission withthat of a reference performance. Such a tool can facilitate the learningby musicians, review of performances, and selection of musicians. Assuch, the systems and methods taught herein can further comprise aguidance module on a non-transitory computer readable medium andoperable as a guidance system that includes a timing comparison tool tocompare the musical submission to the select reference performance, theguidance system providing a feedback that includes a qualitative orquantitative measure of the timing comparison. Likewise, the guidancesystem can include a frequency comparison tool to compare the musicalsubmission to the select reference performance, the guidance systemproviding a feedback that includes a qualitative or quantitative measureof the frequency comparison. Likewise, the guidance system can includeboth a frequency comparison tool and a timing comparison tool providingat least two-dimensions to compare the musical submission to the selectreference performance, the guidance system providing a feedback thatincludes a qualitative or quantitative measure of the frequencycomparison and the timing comparison.

The guidance system can be dynamic, providing an ongoing, continuouscomparison of frequency and timing between a musical submission and it'sreference performance over the course of the performance of the musicalsubmission. In some embodiments, the dynamic guidance module can be on anon-transitory computer readable medium and operable as a dynamicguidance system including a frequency comparison tool and a timingcomparison tool to provide a continuous comparison of the musicalsubmission with the select reference performance from the beginning ofthe performance to the end of the performance, or any section therein,the dynamic guidance system providing a qualitative or quantitativemeasure of the frequency comparison and the timing comparison. In someembodiments, the dynamic guidance module can be on a non-transitorycomputer readable medium and operable as a dynamic guidance system, theguidance system including a frequency comparison tool and a timingcomparison tool for each of the plurality of musicians to continuouslycompare their performance of the musical submission with the selectreference performance while they perform, the dynamic guidance systemproviding a real-time feedback that includes a qualitative orquantitative measure of the frequency comparison and the timingcomparison, the real-time feedback being no slower than 30 millisecondsfrom the occurrence of a respective transient or note performed in themusical submission and used for comparison to the select referenceperformance.

In some embodiments, the term “real-time” can be used to refer to asystem in which the system receives constantly changing audio data fromthe musician and processes it sufficiently rapidly, such that themusician can have some control over the source of the data sent to thesystem. For example, the musician can change the frequency of soundperformed at a particular time in response to feedback from the system.Or, the musician can change the timing at which frequencies areperformed in response to feedback from the system. In some embodiments,the real-time feedback being no slower than 1, 2, 3, 5, 10, 15, 20, 25,30, 50, 75, 100, 150, 200, or 300 milliseconds, or any amount therein inintervals of 1 millisecond milliseconds from the occurrence of arespective transient or note performed in the musical submission andused for comparison to the select reference performance. In someembodiments the real-time feedback ranges in response time from 1-100milliseconds, from 5-50 milliseconds, from 2-20 milliseconds, from 3-30milliseconds, from 4-40 milliseconds, or any range therein in intervalsof 1 millisecond.

Any aspect of the system can operate in real-time, in some embodiments.For example, a user who submits in real-time could get immediatefeedback from the director, or the public through a voting process. Thisprocess would allow for a real-time tutorial in a remote setting, inwhich the musician can respond to feedback as he would in a live sessionat a shared location. A producer could produce a musical performanceremotely in this fashion. Moreover, such tutorials could be done throughthe teachings, presence, performance, or sponsorship of a well-knownmusician, record label, or enterprise such as American Idol, The Voice,or the like. The system could include a means for generating advertisingrevenue through the recognition and users generated through any suchforum that includes the teachings, presence, performance, or sponsorshipof a well-known musician, record label, or enterprise such as AmericanIdol, The Voice, or the like.

The timing reference can be simple, or robust in it's function byserving as a timing interface in that it accounts for tempo changes,complex rhythms and beats, and for that matter, any variable timing thatoccurs throughout a complex musical performance. In some embodiments,the timing reference can effectively “normalize” the timing of aplurality of performances that were performed at locations that wereremote from the location of an administrator, host, director, or anyreviewer that desires to assemble any combination of the plurality ofperformances as an ensemble. In some embodiments, the timing referencecan be selected from the group consisting of a click-track; a drumtrack; a select reference track of a select reference performance,

_(i), the select reference performance having a series of soundfrequencies, each of the frequencies performed at a particular time inthe select reference performance; a residual audio track representing amulti-track digital audio file of a musical work having the selectreference performance removed, the residual audio track having a seriesof sound frequencies, each of the frequencies performed at a particulartime in the residual audio track; and, any combination thereof.

In some embodiments, the submission can be a musical performance thathas a minimal timing reference to allow the musician to perform thesubmission “from scratch” to allow for greater creativity and expressionin the submission. In other words, the submission is not a “replacement”but rather an overlay that can be combined with other submissions basedon the minimal timing reference for the timing interface. The timingreference may be just a chart to follow and, for example, the submissionmay be a submission of a guitar part that doesn't yet exist at all inrelation to the chart. In these embodiments, the timing reference may bea click track with no key for the song, a click track with a key forsong, a click track with chord changes, drums with no key for the song,drums with a key for the song, drums with chord changes, drums with abass track, drums with a rhythm track, drums with a plurality ofinstruments, and perhaps a complete musical work looking for an overlay.

In some embodiments, the timing reference is one of a series ofmulti-track digital audio files available upon request by each of theplurality of musicians; each of the series containing a shared musicalwork, highlighting a select reference performance on a select referencetrack within the musical work. In these embodiments, the timingreference can include T reference performances within the musical work,wherein T=Σ

_(i); and,

_(i)=

₁,

₂,

₃, . . . , or

_(T); a select reference track of the select reference performance,

_(i), the select reference performance having a series of soundfrequencies, each of the frequencies performed at a particular time inthe select reference performance; a residual audio track representingthe multi-track digital audio file having the select referenceperformance removed; and, instructions for execution by the processor.

As shown in FIG. 4C, the methods 400 also include a process for amusician to use to contribute to an ensemble of musical performancesfrom a location that is remote from the location of an administrator ofthe ensemble, such as a director that is creating the ensemble, over acomputer network. In some embodiments, the method the contributing 460includes engaging 461 in the system as a musician, the engaging 461including entering 465 the computer network as the musician, the networkhaving a processor operably connected to a memory on a non-transitorycomputer readable medium; a plurality of musicians, each of theplurality of musicians at a remote location from the director and in anoperable communication with the computer network; a director thatselects a set of performances, Σ

_(ir)≤

_(1r),

_(2r),

_(3r), . . . , +

_(Tr), from the plurality of musicians in creating an ensemble; a seriesof multi-track digital audio files available upon request by each of theplurality of musicians; each of the series containing a shared musicalwork. Each of the series, in fact, highlight a select referenceperformance on a select reference track within the musical work, andincluding (i) T reference performances within the musical work, whereinT=Σ

_(i); and,

_(i)=

₁,

₂,

₃, . . . , or

_(T); (ii) a select reference track of the select reference performance,, the select reference performance having a series of sound frequencies,each of the frequencies performed at a particular time in the selectreference performance; (iii) a residual audio track representing themulti-track digital audio file having the select reference performanceremoved; and, (iv) instructions for execution of each of the series bythe processor. The computer network also includes directions forfollowing 471 by the musician, the directions often provided on aninstruction module on a non-transitory computer readable medium,instructing each of the plurality of musicians to request 473 one of theseries of multi-track digital audio files having the select referenceperformance that they desire to replace with their own musicalsubmission,

_(ir); perform 475 their musical submission at their remote locationfollowing 476 the select reference track, the residual audio track, orthe combination thereof, as a timing interface for integration of theensemble; record 477 their musical submission at their remote locationon a non-transitory computer readable medium; and, upload 478 theirsubmission to the computer network on a non-transitory computer readablemedium to enable the director to select the set of performances from aplurality of submissions uploaded by the plurality of musicians for theintegrating 479 of the set of performances to assemble an ensemble.

In some embodiments, the method includes requesting from the computernetwork one of the series of multi-track digital audio files having theselect reference performance that is to be replaced with a submission,

_(ir); recording the submission on a non-transitory computer readablemedium at a location remote from the director while using the selectreference performance as a guide for the timing of performing each ofthe frequencies, and using the residual audio track as an accompaniment;and, uploading the submission to the database on a non-transitorycomputer readable medium for an interfacing of timing of the submissionwith the timing of a second submission that was recorded on anon-transitory computer readable medium by a second musician at a secondlocation remote from the director while using the select referenceperformance as the timing interface, and using the residual audio trackas an accompaniment. The methods can also include a displaying of thesubmission with the second submission as an ensemble, and the displayingcan be obtained using a display module on a non-transitory computerreadable medium operable to display each performance, and the ensembleof performances on a graphical user interface.

The systems and methods taught herein can include the assembly of bothaudio and video data in the creation of an ensemble. In this way, amusician or reviewer can visualize how a performance appears and soundsas an ensemble. Performers can be swapped-out to view a variety ofdifferent combinations in the selection of a set of performances toassemble as an ensemble, or band. As such, in some embodiments, theensemble is a visual ensemble, and each performance in the set ofperformances includes a video recording of the respective submission fordisplay in the visual ensemble. Likewise, in some embodiments, thesystems can be configured to include an input device operable to receivevideo data on a non-transitory computer readable medium; a video engineembodied in a non-transitory computer readable medium, wherein the videoengine is operable to transform input video data to output video data asa video recording of a musical submission for display in a visualensemble; and, an output module embodied in a non-transitory computerreadable medium, wherein the output module is operable to transmit theintegrated audio data, along with the video data, to an output device,the output device operable to display the visual ensemble. Moreover, avideo display can provide a useful tool for the learning artist to usein learning a piece of music. In some embodiments, the video portionincludes a plurality of instructional views of the instructional artistperforming the musical work for selection by the learning artist.

Any performing artist will appreciate the systems and methods taughtherein. The performing artist in an audiovisual work can be amateur orprofessional. For example, the performing artist can be a popularartist, in some embodiments. Likewise, the musical work can be amateuror professional. For example, the musical work can be a popular musicalwork, in some embodiments. And, in some embodiments, the musical workcan be an original, master multi-track recording of a popular,professional artist or group. As such, the performing artist can even bethe popular professional artist or group that performed the original,master multi-track recording.

The tools provided herein to musicians and administrators are robust, asthe timing reference builds-in a great deal of utility to the systemsand methods. It should be appreciated that the user can isolate anydesired portion of an audio track, and associated video portion,including any one or any series of sounds, beats, and bars. In someembodiments, the desired portion can consist of a single musical note, asingle repeating rhythmic pattern, or a series of accurate and preciseselections from the tempo map. Each selection in the series ofselections can consist of a musical note, a plurality of beats, a bar, aplurality of bars, a repeating rhythmic pattern, or any combinationthereof. The musical work can include any format or compilation that oneof skill would find useful as applied to the teachings herein. In someembodiments, the musical work can be a multi-track musical workcomprising an isolated audio track consisting of a solo performance thatwas not obtained through a process of subtracting the solo performancefrom a mixture of performances. Likewise, in some embodiments, themusical work can be a multi-track musical work comprising a plurality ofisolated audio tracks consisting of a plurality of solo performancesthat were not obtained through a process of subtracting the soloperformances from a mixture of the performances.

In some embodiments, an audiovisual file comprises a video portionsynchronized with an audio track, wherein the video portion includes aperformance of a work by a performing artist. In these embodiments, adynamic point referencing method can be included within the musical workfor an accurate and precise selection and cycling of a desired portionof the synchronized audio and video portions by the learning artist. Thedynamic point referencing can include a tempo map of the musical workthat is adaptable for a plurality of tempos within the musical work. Theaccuracy and precision of the selection of the desired portion of thesynchronized audio and video data provides an instructional tool that isvaluable to the learning artist. For example, a dynamic pointreferencing of the musical work can provide accuracy and precision inthe selection of the desired portion when using the methods providedherein. The tempo map can include a plurality of tempos or a complextempo, in some embodiments. It should also be appreciated that the tempomap can represent a fixed tempo in some embodiments, such that the tempomap comprises a single tempo in a substantial portion of a work, andeven consists of a single tempo throughout the work.

The systems and methods can be operable using a hand-held device, insome embodiments. In some embodiments, the systems are operable tofunction as a particular machine or apparatus having the additionalfunction of telecommunications, word processing, or other forms ofgaming. Likewise, the system can be operable to function as a particularmachine or apparatus not having other substantial functions. Thehandheld apparatus can be a smart phone, iPAD, laptop computer, oranother type of handheld computing device having other substantialfunctions.

FIG. 5 illustrates a circuit diagram of the system, according to someembodiments. The solo audio part (the part to be learned) is retained inan audio store 505, and the audio recording of the other parts of thecomposition are retained in an audio store 510. Storage areas 505 and510 may reside in separate devices or in a single storage device, butcan be accessed separately. The progressive retrieval of audioinformation from stores 505 and 510 is controlled by a playbackcontroller 515 such that the various parts of the composition becomesynchronized in time. The solo audio signal can pass through a variablegain element 520 so that it's volume level in an overall output 540 canbe controlled. Similarly, the other audio signals pass through a gainelement 525 and are fed to a mixing element 530, to be combined with thesolo instrument audio signal provided from gain element 520. A similarcircuit diagram can be used for video stores.

An external instrument input element 545 enables an external audiosource, such as the practice instrument, to be included in overalloutput 540. The signal provided by instrument input element 545 passesthrough a gain element 550 before delivery to mixing element 530.

The overall output level can be controlled using a gain element 535,which receives an input signal from mixing element 530 and feeds anoutput signal to overall output 540. The various gain elements can becontrolled directly through user controls or through signals from acontrolling device such as a microprocessor. In some embodiments, otherparts of the musical performance may be stored in separated storageareas to facilitate the learning process or to accommodate multipleplayers.

FIG. 6 illustrates a circuit diagram of the system including a clickfunction, according to some embodiments. An audible timing reference, ordynamic point reference, is included into the signal provided to theoutput of the device. This timing reference may be a periodic sound suchas a click, beep, or a tone, which can be recorded and stored in amethod similar to that used for the other audio signals, or which may besynthesized by an audio signal generator 605 while the musical piece isbeing played. This additional sound is delivered to mixing element 530via a variable gain element 610, which allows the audible level of thedynamic point reference to be adjusted.

In some embodiments, the dynamic point reference can be created from atempo map, such that the dynamic point reference, or indexing, isvariable over time, rather than constant. Many musical works cannot beaccurately mapped using a constant timing reference, as these works canbe found to have, for example, variable and unpredictable tempo changesduring the course of the performance of the musical work. And, this isparticularly true of original musical works, such as those found in theoriginal multi-track master recordings that are the subject of manyembodiments of the teachings provided herein.

Sheet music, for example, can be obtained for such original musicalworks, and these are generally expected to comprise transcriptions of anoriginal and complex variance of tempos into a constant tempo. Theteachings provided herein, however, do not use a constant tempo but,rather, are based on a custom tempo mapping, for example, of an originalrecording providing a dynamic point reference that is substantiallyidentical to the originally recorded work. Traditional sheet music, ordigital sheet music (e.g., pdf scans of sheet music), of popular musicprovide a derivative musical work that is, in most cases, transcribedmuch different than the originally recorded musical work. Traditionalsheet music, for example, generally uses repeated sections that ignorethe nuances that occur in an original musical work. The teachingsprovided herein, however, do not use such repeated sections, but ratherare linear and verbatim representations of the original work transcribedfrom the original, isolated instrument tracks of an original masterproviding a transcription of the work that is transcribed to besubstantially identical to the originally recorded work.

FIGS. 7A and 7B illustrate a musical notation display designed for usewith the system, according to some embodiments. In some embodiments, adisplay provides an animated graphical representation of the musicalnotation relative to the part to be learned, whereby such graphicalrepresentation scrolls along the display area in synchronism with theaudio music signal. In FIG. 7A, for example, a time region 705,710 ofthe display area is marked to identify the present moment, with thegraphical representation of the music moving past time region 705,710 asthe music part is being played. By this arrangement, the display areaenables a player to see musical events in advance that are immediatelyfollowing the present moment. In some embodiments, a recent musicalnotation may also be seen in retrospect. In FIG. 7B, for example, adesired portion of the musical work is accurately and precisely selectedfor cycling, as shown by the broken line, in which the speed of thecycled portion can be adjusted for the skill level of the user.Moreover, the size of the broken line box can be reduced to a very highresolution to capture nearly any desired time increment in the work, forexample, a small set of beats, a single beat, a fraction of beat, andthe like. Accordingly, the apparatus and methods of presenting musicalnotation are advantageous for at least the reason that a user isprovided with a temporal context, tempo speed adjustments, and videomatching of the same for the musical events of interest.

Moreover, and contrary to traditional sheet music notation, thescrolling of the music can be continuous, removing the need for pageturns. And, as described herein, the scrolling of the notation canreflect the variable and unpredictable tempo changes of an original,multi-track master, rather than a derivative work produced using aconstant, or substantially constant, tempo. Having the function ofdigital indexing with the tempo map, as described herein, allows for anaccurate and precise selection of the desired portion of the musicalwork, as well as an accurate and precise cycling at any tempo desired bythe user. A tempo can be referred to as “substantially constant,” forexample, where it was not created to reflect the variable andunpredictable tempo changes of the original, multi-track master. Itshould be appreciated that a transcriptionist of music could vary tempofrom time to time in a musical work, but one of skill would considersuch tempo changes to be substantially constant in view of the variableand unpredictable tempo changes that can occur from the originalperformance by the original artist as found in an original, multi-trackmaster recording.

As shown in FIG. 8A, the system 800 can include an instruction for anyinstrument, such as a guitar 802, and performance can be rated 872. Thesystem 800 can include a video camera feature 861 that allows users tovideo themselves playing and superimpose or compare the user capturedvideo to instructional video, and this feature 861 can also generateanimations of the user, in some embodiments, using a motion-capturecamera. Viewing angle adjustment 804, and a panning feature 806, can bea standard feature. In some embodiments, the system includes a narrativevideo/audio track through a narrative camera function 862 that gives theuser information or instruction pertaining to the song. In someembodiments, the system includes a narrative teacher feature 863 toselect from a variety of different narrative teachers that may includecelebrities or popular artists. Moreover, the system may also have anaccuracy feature 864 that automatically compares the audio/video from auser's performance to the instructional audio/video as a measure of theuser's performance. The accuracy feature 864 can even serve as real timefeature that notifies a user when a wrong note has been performed.

In some embodiments, the system includes an option that stops the userwhen he makes a mistake. In some embodiments, a tolerance option 865 candefine the amount of tolerance that system will allow before stoppingthe user or indicating that a mistake has been made. And, in someembodiments, a multi-channel playback engine would let the user isolateand balance the audio level of the instructional instrument, theresidual component track, the click track and any other instruments thatmay be isolated within the session.

In some embodiments, the system includes a click track 866 that helpsthe user keep time with the song. And, in some embodiments, the system800 has an ability to provide a visual representation 867 of thesuggested finger placement on instrument (i.e. #1 corresponds to thumb,#2 index figures, etc.). Moreover, the system can have an a notationenabling function 868 to provide the user with a notation screen as wellas a video screen as an additional learning tool.

In some embodiments, the system has a note to note function 869, withthe capability for users to advance note to note (set to a specific notevalue—⅛, 16th, quarter, half notes, etc.) at their own pace using a stepadvance mode. This would allow the learning of difficult musicalpassages by viewing the displays and hearing the notes in a non-realtime “move-on when you're ready” method.

The system could also include a plurality of communities or socialnetworks that include like-minded musicians or fans that can comparetheir skill level or exchange tips and lessons. As shown in FIG. 8A, aplayer may submit and share through a community share function 871,sharing a recorded performance 872 with the community and display thegrading or skill level that has been achieved on a per song basis or anoverall average ability rating. In some embodiments, the performance maybe graded by the host or celebrity narrative chosen.

A user could have a video chat enabling feature 873 as an access to theonline community to get direct feedback or tips from other usersutilizing a video chat system. And, in some embodiments, a player mayreceive points for his performance and achieve a level of status in thecommunity. The system could also be designed such that a player mayexchange his points with other members or redeem his points for avariety of goods or services within the community or the company. Insome embodiments, a player may be rewarded or graded on his appearanceas it pertains to certain communities or predetermined genres.

In some embodiments, the system could include an instrument soundmodeling option 874 that could synchronize a sound modeling processor orfx processor that matches the tone of the chosen song and automaticallyadjusts and changes with the unique tonality of each particular sectionof a song. And, in some embodiments, the system could also give theusers feedback on the quality or their sound as it pertains to theirchosen instrument or the selected song.

The systems taught herein can be in a device that can be configured foruse with any musical instrument, for example, guitar, piano, drums, or avocals. For example, such a device can be configured for use with apiano. The device can be substantially limited to the teachings hereinand include a housing or container of any shape, such as a shapedesigned to sit on top of a piano in place of standard sheet music. Amusic display may be included as a liquid crystal display (LCD) screenor other type of display screen, and one or more transport buttons suchas, for example, a play button, a stop button, and a pause button, canbe provided. Such a device can comprise a number of controls, which maybe configured as knobs or other similar state selectors known in theart. The device can include a first group of state selectors that relateto the “blender” function of the device and are structured to controlthe user's input, the music without the piano contribution, and thepiano itself. A second group of state selectors can be provided torelate to the “master” function of the device and control volume, clicktrack, and tempo. A third group of state selectors can be provided tocontrol the on/off function of the device and may include indicatorlights, light intensity control, and additional playback controls, forexample. The device can also include one or more speakers, a soundmodule for electric pianos, and one or more ports for connecting thedevice to other units such as, for example, USB ports, phono jacks, andpower jacks, or perhaps musical instruments, such as electric organs andguitars, for example. In some embodiments, a USB port may be used toconnect the device to a computer system. In some embodiments, forexample, the USB port allows for downloading of audio data to a largercomputer memory storage location. In some embodiments, data may also besupplied to the device and and/or stored in removable data memory cards.

In some embodiments, the multi-track audio files are produced fromoriginal multi-track recordings, and these recordings may originate onanalog tape, such as analog multi-track tape (e.g. 1 track to 24tracks), a digital tape format (e.g. pulse code modulation, PCM, digitaltape format). In some embodiments, an analog tape format is firsttransformed into a digital recording and a multi-track audio file isproduced from the digital recording. In some embodiments, the originalmix is recreated by taking all of the different tracks and making themix to simulate the original recording. The mixing can be a manualprocess and can be done with an analog console, new digital console, orthe mix can be done on a computer using basically any mixing techniqueknown to one of skill. In some embodiments, older analog tapes need tobe restored, such as by a baking procedure, before attempting arecreation.

It should be appreciated that the teachings can apply to any piece ofmusic containing virtually any musical instrument including, but notlimited to string instruments, brass instruments, woodwind instruments,percussion instruments, and vocals. In some embodiments, pieces of musichaving variable tempos, rhythms, and beats can be learned with more easedue to the manually created and variable metronome function, as well asthe manually created audio files having superior isolated instrumentaudio track quality. In some embodiments, songs are complicated and havechanging or otherwise un-isolatable beats that would be more difficultto learn without the teachings provided herein.

The user will often have a set of multi-track audio files to learn onthe system.

FIG. 10 illustrates a display for a graphical user interface offeringselections within a track list of a multi-track audio recording setdesigned for use with the system, according to some embodiments. TrackList Display 1000 shows the selections in a multi-track audio recordingset or track list 1050. The user chooses an audio file and proceeds tothe menu bar 1005 to select a function from home page 1010, volume/faderpage 1015, music transcription and tablature 1020, cycling page 1025,and help page 1030.

The user can access an information page regarding the piece of musicselected from the track list, such as the band, song title, album,tempo, and tuning. FIG. 11 illustrates a display for a graphical userinterface offering information on the track list selection in thesystem, according to some embodiments. Information Display 1100 providesthe user with the selection bibliographic information 1105 containinginformation on the band, song title, and album, and selection technicalinformation 1110 provides information on the tempo and tuning for theselection. Play function 1150 allows the user to begin learning theselection.

The volume/fader page 1015 has several functions that enables a user toeffectively learn and mix music. FIGS. 12-15 show variousfunctionalities of the volume/fader page 1015. FIG. 12 illustrates adisplay for a graphical user interface offering state selections fortrack faders, volume fader control, and transport selections, in whichall tracks are selected on a multi-track audio recording designed foruse with the system, according to some embodiments.

Volume/fader display 1200 provides the functionality of track on/off(i.e. mute) control 1205 for each track file. The functionality of thevolume/fader controls 1210 is provided by the volume indicators 1215 toindicate sound pressure level and the faders 1220,1225,1230,1235 toadjust volume in the manner of a potentiometer or digitometer, forexample. Transport section 1250 provides a time bar 1255 to indicate aposition in the piece of music and can also contain markings, such ascolors, to indicate the intro, pre-chorus, verse, solo, bridge, outro,chorus, and middle section, for example.

The transport section 1250 also provides several state selectionfunctions: a rewind 1260, pause 1265, fast forward 1270, stop 1275 asnormal transport control state settings; cycle 1280 to allow a user torepeat a desired section of the piece of music; and slow 1285 to allowthe user to slow the song by a predetermined amount and serve the useras a function commonly known as a speed control, tempo adjustment, ortempo control. In some embodiments, the slow 1285 function can be adefault setting (e.g. to slow by some percentage, such as 50%, 75%, orthe like) and, in some embodiments, the user can define a desired speedsetting. As such, in some embodiments, the playback controls can be tothose found on about any tape deck or video cassette recorder, such as“previous”, “next”, “play”, “pause”, and “record”. And, in someembodiments the playback controls include a “cycling” function to allowthe user to cycle a particular section or measure over and over againuntil the user is satisfied in the learning experience.

In some embodiments, the transport section 1250 can be operable within asingle functioning section of a system, such as a single page of asoftware application. In some embodiments, the transport section 1250can be operable within a plurality of functioning sections of a system,such as in a plurality of pages of a software application, such that thetransporting can occur quickly at the choosing of the user withouthaving to move from one section of the system to another section of thesystem. A music label and timer 1290 is also provided as a referencedata point for the user.

In some embodiments, the mixing of audio can be handled on an individualsample per track basis. Each track can be represented individually,having to keep track of its own samples, duration, levels and peaks,gain, and time. Once each track can be initialized and loaded out of itsfile, and then handed off to a subsystem to decode the compression andmake samples. An example of a subsystem can include, for example Apple'sCOREAUDIO subsystem. After the samples are made available, the track canthen be assigned to a master track handler object referred to as themixer object, and saved recorded audio sessions can be loaded at thistime as well. The mixer object primes the subsystem and initializes theoutput. In these embodiments, the touching of the “play” button can beused to start the mixer in its function of combining the audio withinbuffers, where the mixer calls out to each track asking it for its nextframe of audio. The audio frames can be added to the playback buffer andenqueued, and all timing on the audio can then be synced to allow theaudio to be synced to a subnote level to reduce or eliminate creepwithin tracks.

In some embodiments, the audio recording can be handled by a mannersimilar to the individual audio playback, where extensions to record toa file and store its place within a song can be used. Once the userclicks a record function, a recorder object can be built, and the objectcan then start a file and initialize the recording. Once initialized,the recording class can store the current play time within the song tothe measure and begin. The user can tell the device to stop recording,and the object can then mark that time and store the duration of thedata stream in a settings file. The audio data is then flushed to thefile, a track handler object can then be created with its start time andduration set, and the mixer can be updated to allow future playback ofthe recorded audio along with the rest of the prerecorded audio.

FIG. 13 illustrates a display for a graphical user interface offeringstate selections for track faders, volume fader control, and transportselections, in which all tracks but the click track are selected on amulti-track audio recording designed for use with the system, accordingto some embodiments. FIG. 14 illustrates a display for a graphical userinterface offering state selections for track faders, volume fadercontrol, and transport selections, in which only the isolated instrumentaudio track and the emulated instrument audio track might be selected ona multi-track audio recording designed for use with the system,according to some embodiments. FIG. 15 illustrates a display for agraphical user interface offering state selections for track faders,volume fader control, and transport selections, in which only theresidual track and the emulated instrument audio track are selected on amulti-track audio recording designed for use with the system, accordingto some embodiments.

A user can benefit by reading music while playing. FIG. 16 illustrates adisplay for a graphical user interface offering musical transcriptionand tablature audio data, in which only the current section of music isshown for a multi-track audio recording designed for use with thesystem, according to some embodiments. Music transcription and tablaturepage 1600 provides the current transcription and tablature 1605 in abright display and the upcoming transcription and tablature 1610 in adim display, where the current represents the music currently playing,and the upcoming represents the music immediately following the musiccurrently playing. Each measure, for example, can have a timecode andduration that represents where in the song's timeline that measure isplayed. Using that information along with the song's current playbackposition, the tablature can be synchronized along with any playingaudio. Each measure can be designed to display notes or chords and whichstrings or frets to use. In some embodiments, a user may desire astandard music scale for display. The musical transcription andtablature can also be a dynamic, scrolling display, in some embodiments.

A user can also benefit from a function that allows for an isolation ofa particular section of a piece of music in an audiovisual work, as wellas a cycling of that section to enable the user to focus and practice onthat section of the music. FIG. 17 illustrates a display for a graphicaluser interface offering selection of a section of a piece of music bybar or set of bars in a multi-track audio recording designed for usewith the system, according to some embodiments. Cycling page 1700displays section 1705 of the digital audio file, and bar 1710 or bars1715, for example, can be isolated and selected for playback by theuser. The user can then use the methods taught herein to focus and learnthe particular section 1705, bar 1710, or bars 1715, for example. Assuch, in some embodiments, the playback can also be controlled through a“reel screen”, where every measure is segmented. In such embodiments,any audio the user has recorded can also be displayed on the screen inthe measures in which it exists to allow the user to quickly find thataudio and listen to their play of that session as graphed over theoriginal audio data. In some embodiments, a “tape” can be graphicallydisplayed to show the markings of the playback tracking bar. And,consistent with the teachings herein, a video display can complement thesystem, in some embodiments.

The user may have questions, and as such, a help page is always useful.FIG. 18 illustrates a display for a graphical user interface offering ahelp page for the system, according to some embodiments. Help page 1800is a simple depiction of information that a user can obtain from thesystem.

FIG. 19 shows how a network may be used for the system, according tosome embodiments. FIG. 19 shows several computer systems coupledtogether through a network 1905, such as the internet, along with acellular network and related cellular devices. The term “internet” asused herein refers to a network of networks which uses certainprotocols, such as the TCP/IP protocol, and possibly other protocolssuch as the hypertext transfer protocol (HTTP) for hypertext markuplanguage (HTML) documents that make up the world wide web (web). Thephysical connections of the internet and the protocols and communicationprocedures of the internet are well known to those of skill in the art.

Access to the internet 1905 is typically provided by internet serviceproviders (ISP), such as the ISPs 1910 and 1915. Users on clientsystems, such as client computer systems 1930, 1950, and 1960 obtainaccess to the internet through the internet service providers, such asISPs 1910 and 1915. Access to the internet allows users of the clientcomputer systems to exchange information, receive and send e-mails, andview documents, such as documents which have been prepared in the HTMLformat. These documents are often provided by web servers, such as webserver 1920 which is considered to be “on” the internet. Often these webservers are provided by the ISPs, such as ISP 1910, although a computersystem can be set up and connected to the internet without that systemalso being an ISP.

The web server 1920 is typically at least one computer system whichoperates as a server computer system and is configured to operate withthe protocols of the world wide web and is coupled to the internet.Optionally, the web server 1920 can be part of an ISP which providesaccess to the internet for client systems. The web server 1920 is showncoupled to the server computer system 1925 which itself is coupled toweb content 1995, which can be considered a form of a media database.While two computer systems 1920 and 1925 are shown in FIG. 19, the webserver system 1920 and the server computer system 1925 can be onecomputer system having different software components providing the webserver functionality and the server functionality provided by the servercomputer system 1925 which will be described further below.

Cellular network interface 1943 provides an interface between a cellularnetwork and corresponding cellular devices 1944, 1946 and 1948 on oneside, and network 1905 on the other side. Thus cellular devices 1944,1946 and 1948, which may be personal devices including cellulartelephones, two-way pagers, personal digital assistants or other similardevices, may connect with network 1905 and exchange information such asemail, content, or HTTP-formatted data, for example. Cellular networkinterface 1943 is coupled to computer 1940, which communicates withnetwork 1905 through modem interface 1945. Computer 1940 may be apersonal computer, server computer or the like, and serves as a gateway.Thus, computer 1940 may be similar to client computers 1950 and 1960 orto gateway computer 1975, for example. Software or content may then beuploaded or downloaded through the connection provided by interface1943, computer 1940 and modem 1945.

Client computer systems 1930, 1950, and 1960 can each, with theappropriate web browsing software, view HTML pages provided by the webserver 1920. The ISP 1910 provides internet connectivity to the clientcomputer system 1930 through the modem interface 1935 which can beconsidered part of the client computer system 1930. The client computersystem can be a personal computer system, a network computer, a web TVsystem, or other such computer system.

Similarly, the ISP 1915 provides internet connectivity for clientsystems 1950 and 1960, although as shown in FIG. 19, the connections arenot the same as for more directly connected computer systems. Clientcomputer systems 1950 and 1960 are part of a LAN coupled through agateway computer 1975. While FIG. 19 shows the interfaces 1935 and 1945as generically as a “modem,” each of these interfaces can be an analogmodem, isdn modem, cable modem, satellite transmission interface (e.g.“direct PC”), or other interfaces for coupling a computer system toother computer systems.

Client computer systems 1950 and 1960 are coupled to a LAN 1970 throughnetwork interfaces 1955 and 1965, which can be ethernet network or othernetwork interfaces. The LAN 1970 is also coupled to a gateway computersystem 1975 which can provide firewall and other internet relatedservices for the local area network. This gateway computer system 1975is coupled to the ISP 1915 to provide internet connectivity to theclient computer systems 1950 and 1960. The gateway computer system 1975can be a conventional server computer system. Also, the web serversystem 1920 can be a conventional server computer system. Alternatively,a server computer system 1980 can be directly coupled to the LAN 1970through a network interface 1985 to provide files 1990 and otherservices to the clients 1950, 1960, without the need to connect to theinternet through the gateway system 1975.

Through the use of such a network, for example, the system can alsoprovide an element of social networking, and also a gaming platform,whereby users can contact other users having similar subject-profilesand compete with one another qualitative and/or quantitatively, usingsubject scores or actual numerical scores. In some embodiments, thesystem can include a messaging module operable to deliver notificationsvia email, SMS, and other mediums. In some embodiments, the system isaccessible through a portable, single unit device and, in someembodiments, the input device, the graphical user interface, or both, isprovided through a portable, single unit device. In some embodiments,the portable, single unit device is a hand-held device. In someembodiments, the systems and methods can operate from the server to auser, from the user to a server, from a user to a user, from a user to aplurality of users, in an MMO environment, from a user to a server to auser, from a server to a user (or plurality of users) and a teacher (orplurality of teachers), or a server to a plurality of users and aconductor, for example. The teachers or conductors can be eitherreal-time teachers or conductors, simulated teachers or conductors, orrecorded teachers or conductors, in some embodiments. The network canprovide text and/or audio for real-time messaging, posting of messages,posting of instructional, posting of news or other items of a relatedinterest to the users, and the like.

The systems and methods taught herein include the use of the computersystem within any larger network system or method. It should also beappreciated that a network can include traditional network media. Forexample, television cable, optical fiber, satellite, and the like, inwhich such digital information can be transmitted to users. Such use oftraditional media will facilitate use of the teachings provided hereinwith traditional media entertainment channels, such as reality TV,amateur talent shows and competitions, and the like.

The following examples are illustrative of the uses of the presentteachings. It should be appreciated that the examples are for purposesof illustration and are not to be construed as otherwise limiting to theteachings.

Example 1. Selecting and Assembling a Visual Ensemble of PerformancesPerformed and Uploaded to the Computer Network to Create a BandPerformance

Any genre of music performances can be assembled using the teachingsprovided herein. Audiovisual performances can be submitted to a computernetwork and assembled to create a ensemble, or band performance. Thiscan be a powerful tool to assemble performances that were uploaded assubmissions from remote locations, perhaps even worldwide, over acomputer network. The submissions are received and reviewed over thenetwork, as well as selected and assembled for display as the visualensemble. For example, a church can request auditions online for a newchurch band, seeking out particular musicians, such as vocalists. And,the auditions can be performed at remote locations, such that the entireprocess can be done comfortably, as it does not require scheduling,meeting, and attending numerous auditions at a particular location. Thechurch can receive submissions, review submissions, select and assemblea set of performances, and create and display a visual ensemble of themusical performances over a computer network to select members of thenew church band.

The Audition System

The audition system can include a computer network having a director, aswell as a processor operably connected to a memory that includes adatabase on a non-transitory computer readable medium. The computernetwork will audition a plurality of musicians, each at a remotelocation from the director and in an operable communication with thecomputer network.

The church can select a particular musical work for the audition, thework containing individual tracks that isolate musicians playinginstruments, vocal or otherwise, that exemplify the type of content andtalent desired for the church band. The director may select the musicalwork to test the skills of the musicians, as opposed to merely selectinga song that the director intends the band to play. Any number ofmusicians, including singers, can respond to a request for auditions,and they can download the musical work to emulate the musical work,perhaps using audible and visual click tracks, voice over guide tracks,or a synchronized video having any type of notes, lyrics, song maps orany other information to learn and practice the musical work untilproficient for recording and submission. The methodology of learning,recording, and submitting could also include a plurality of videosshowing human or virtual instruction, such as animation of the movementof body parts for playing a particular instrument. This could alsoinclude the position of an implement, such as sticks, mallets, pedalswith fx or controller values, hand held bows for violin, cello, viola,etc., or any other implement that is physical. The instruction caninclude any additional electronics for instruction known to one ofskill, such as artificial intelligence or virtual presence, including ahologram display or equivalent.

Receiving the Musical Work and Recording the Submission

The musical work can be presented to the plurality of musicians as anoffering of a series of multi-track digital audio files; each of theseries sharing the entirety of the musical work, yet highlighting aselect instrument performance within the musical work. The musical workcan have, for example, (i) T reference instrument performances withinthe musical work, wherein T=Σ

_(i); and,

_(i)=

₁,

₂,

₃, . . . , or

_(T); (ii) an isolated audio track of the select reference performance,

_(i), the isolated instrument audio track having a pitch and a timing;(iii) a residual audio track representing the multi-track digital audiofile having the select reference performance removed; and, (iv)instructions for execution by the processor of the computer network.

The plurality of musicians auditioning should be instructed to eachrequest one of the series of multi-track digital audio files having theisolated instrument audio track that is to be replaced with their ownsubmission,

_(ir), in the audition. They should be asked to record the submission attheir location, remote from the director, on a non-transitory computerreadable medium for uploading to the computer network. They should alsobe asked to follow the frequency and the timing of the isolated audiotrack as a guide, since the evaluation will involve comparing theirperformance to the reference performance. They can also be instructed touse the residual audio track as an accompaniment to hone their skills.Once they're satisfied with their performance, they can upload asubmission of their performance to the database.

To record the submission, the musician can merely push a “video record”button on the system and perform. Once completed, the musician can beprompted to delete the residual background music and submit only theperformance to reduce file size, since the common denominator betweenall submissions for a particular instrument will be the same residualbackground following the same frequency and timing to serve as a timinginterface between all performances. The musician can request a real-timeprofessional feedback as they perform or after the recording. Forexample, the system may include an “expert analysis” module whichprovides a performance rating in real-time, and can also provide a moredetailed report after the recording has stopped. The expert analysismodule can offer detailed information about the performance at any pointin time within a performance, in detail, with recommendations through aprofessional live teacher or delayed after review by a professional liveteacher.

The Selection Process

The best musicians for each particular instrument can be chosen usingany one, or any combination, of methods. For example, the best musicianscan be chosen by the director, by votes from professional reviewers, byvotes from peers, by votes from the public, or by votes from anyonehaving access to a vote through the computer network. Any method ofgaining access to a vote can be used. Access to a vote can be given, forexample, by lottery, by demographic, by selection due to profession, byvote, by sponsorship, or by paying to place a vote. The best musicianmay also be chosen by the expert analysis module. The expert analysismodule can be adjusted to a default setting that evaluates the musicalwork using a default frequency and timing parameter, or it can be set toa preferential setting for a particular audition, either for a completemusical work, for sections of the work, or vary in settings by section.Some sections may allow for more variability in performance of frequencyand timing because the sections are “loose”. Other sections may requireless variability in performance of frequency and timing because thesections are relatively “tight”. The manner in which a performance offrequency and timing conforms to the musical work will determine whetherthe performance is “in the pocket.” And, the allowable variation inperformance from the musical work defines the size of the pocket.

The next step is the selecting of a set of performances, Σ

_(ir)≤

_(1r),

_(2r),

_(3r), . . . , +

_(Tr), from the plurality of submissions for display through thenetwork; and, displaying each of the set of performances over thenetwork alone or in combination with other submissions as a visualensemble. Each of the top musicians selected for the ensemble can bepresented as a set for display, or an even larger set that includesrunners-up, or perhaps even all submission, can be configured forselection and display alternatively. A listener of the ensemble merelyneeds to activate the start button and the ensemble will play. Thisallows a listener, or reviewer, to listen to any one, or any combinationof performances, including the set of performances selected by thedirector, votes, etc. In this manner, the director, listener, choirmaster, etc., can also use a transport selector to move to differentsections of a song, loop the section, and change the relative volumebetween the different musicians. These functions can operate in realtime, or while the music is stopped, to make a desirable balance. Thesystem can also include an auto-balance module to find a desirablebalance between musicians. As each of the musicians are alternated,favorites can be selected and stored in real-time, or when the music isstopped, to mix and match players in the selection of the band.

Example 2. Custom Mapping for the Timing of the Musical Work

One of skill in the art of music production will understand that mappinga musical work is a skill obtained through experience and is not easy.There's nothing obvious about mapping, and there's no existing systemthat offers the custom mapping approach to assembling remotely-performedand remotely-uploaded musical performances as an ensemble. And, thismapping, although complex, can be used to provide a control for thetransport of the musician to the desired portions of the musical work.

The easiest example of mapping can include the use of a transient orbeat, for example, with drums, from the start to the end of a musicalwork. Once this type of mapping is done, the map can be expressed as atiming reference in any form of electrical signal that is audible orvisual, or both. Or, the beat of the drums themselves can be used as thetiming reference. The skilled artisan could identify a first beat andmap it with subsequent beats, such as a first kick, then a first snare,a second kick, and then a second snare, representing one bar in quarternotes. That's a simple tempo map.

The tempo map is based on “distance” that is measured in time betweenbars, or sometimes between beats, e.g., in milliseconds. The distancebetween bar 1 and bar 2 could be, for example. 119.5 beats per minute,between bar 2 and bar 3 could then be 119.38 beats per minute, where theskilled artisan can capture subtle fluctuations in tempo. The capturingof such fluctuations, whether subtle or not, allows for a mapping thatallows a learning artist to locate exactly where the beat is, e.g. adownbeat or an upbeat.

What if there are no drums? For this purpose, the skilled artisan coulduse another instrument, such as a guitar, in search of a downbeat andupbeat, for example. Many musical transients may provide a means foridentifying quarter notes, eighth notes, etc, using any point ofreference known to one of skill, and the choice depends on thecomplexity of the work or the tempo.

What if no tempo at all for some time, such as 8 bars? In this case, theskilled artisan could take the last beat of a section and the first beatof a following section and measure the distance between the selectedbeats.

What if song starts with an ad-lib vocal with no time reference at all?In this case, the skilled artisan could look at the overall tempo of asong, and then use expert interpretation to select a tempo, followed byan iterative process that includes, for example, manually shuffling,adjusting, and laying-out the map.

Tempo mapping functions that are available through PROTOOLS or othersoftware programs are inadequate for the uses taught herein, as the donot provide preconfigured custom tempo mapping sets that will easily andautomatically align to any given song. Our predefined custom tempo mapsprovide a technology for obtaining an accuracy and precision in themapping of a reference performance that is not otherwise available tothe musician.

Example 3. A Guidance System For Use by (i) an Auditioner in Preparationto Submit a Performance; and, For Use by (ii) a Reviewer in Evaluating aPerformance Against a Reference

A musical performance includes the execution of a series of soundfrequencies by a musician, each of the series performed by the musicianat a particular time in the musical performance. Although a precise andaccurate performance of each of the frequencies at their respectivetimes represents only two of several variables in a great performance,these two variables provide the basis for a guidance system that can beused by the musician in the development of a performance, or by areviewer of the performance in comparing the performance to a referenceperformance.

It should be appreciated that a musician can follow a timing referenceto perform in a desired manner. A musical work can have complex timing,for example, due to following a live drummer playing without a timingreference, or a beat can slowly drift out-of-time because it wasrecorded on a tape machine that slowly changed speed over time. In fact,the musical work may have been cut to the timing of the drummer's liveperformance rather than any click track that may have been printed onthe tape machine. The timing reference, for example, might be selectedand provided by a director for a particular set of auditions. Twoperformances that follow the same timing references and were recorded atlocations remote from one another can be uploaded to a computer networkand assembled as an ensemble using the shared timing reference as atiming interface. This is of particular importance when combiningperformances that consist of variable tempos, which includes mostperformances. The timing reference can be, for example, (i) aclick-track; (ii) a drum track; (iii) a reference track of a selectreference performance performed with or without any other accompanimentand having a series of sound frequencies, each of the frequenciesperformed at a particular time in the select reference performance; (iv)a residual audio track representing a multi-track digital audio file ofa musical work having the select reference performance removed, theresidual audio track having a series of sound frequencies, each of thefrequencies performed at a particular time in the residual audio track;or, (v) a combination thereof.

Using the teachings provided herein, a guidance system can be configuredto display an independent analyses of the frequencies performed by amusician, as well as an independent analysis of the timing of theperformance of the frequencies. Both analyses can be combined on a chartthat the musician can follow when performing the musical work, or whenreviewing the performance. As such, a reviewer of performances canlikewise use the guidance system to evaluate the performance. In fact,the guidance system can be used in the review of a plurality ofperformances as a rating tool. The chart can include a notated, digitaltranscription having absolute values of frequency and/or timingcorrelated with that of the timing reference. As such, the systemprovides valuable information about a musical performance, suchinformation never-before available to a musician performing a selectreference performance, or to a director or other reviewer of theperformance.

Frequency, Timing, and Correction of Anomalies

It is not a simple task to emulate the series of frequencies and timingof the performance of each of the frequencies in a referenceperformance, and this is generally true regardless of the instrumentyou're playing. As such, both the musician and administrator can benefitfrom a tool that provides timing and/or frequency comparisons, comparingthe musician's performance to a template, such as the tempo map designedfor a reference performance. Using at least the teachings set-forthbelow, the tool can also provide a gaming platform, whereby users cancontact other users having similar subject-profiles and compete with oneanother qualitative and/or quantitatively, using subjective scores oractual numerical scores obtained from the guidance tool and guidancemodule, for example.

Some instruments are more difficult and variable than others, however.Certain instruments, such as a violin or fretless bass, for example,have a design that doesn't indicate exactly where a musician needs toposition fingers to play a particular note that is “in tune” (i.e., plusor minus 2 or 3 “cents”). An instrument having a “fixed tuning”, such asa piano, can be tuned to a standard 12 semitones per octave. Suchinstruments are based on a 12-tone scale and are not typically designedto generate notes or micro-tunings in-between the 12 semitones peroctave. A frequency analysis can be used with such instruments, forexample, to show whether a performed note has the correct or incorrectfrequency and provide a measure of the number of semitones between thecorrect note and the performed note as a “correction factor” for themusician or reviewer to use as a quantitative measure of the accuracyand precision of the performance.

The musician or reviewer can simply use the correction factor for aqualitative and/or quantitative measure of a comparison of the musiciansperformance to the reference performance. However, the referenceperformance can contain undesirable anomalies, such as errors in one ormore frequencies performed in the reference, and/or the time at whichthe one or more frequencies were performed. As such, the system canidentify these anomalies and replace them with a representation of theone or more frequencies and timing that would be expected to be presentin a desirable performance. These can be identified by creating anotated, programmed track, a track tempo-mapped to serve as the timingreference for use in the reference track of the system.

For example, each note performed by a musician could be compareddirectly to the timing of a reference performance that was tempo-mappedto serve as the timing reference. If the reference performance happensto “rush” or “drag”, meaning that it plays a little faster (“rushing”)or a little slower (“dragging”) at times than what would be expected ina perfect performance referenced to the tempo map, data representing theperfect performance could be used to replace the anomalies wheredesired. These “rigid” timing inserts could be added, for example, in anentire section, a bar, half bar, or just a few beats, to bring the partin question “back into the pocket”. In this way, a custom compilationcould be configured to represent the “feel” of the live track to theextent desired and correct for anomalies through rigid inserts from theinterpolated “perfect performance”. These custom configurations arepossible through the tempo mapping, a method of “quantizing” aperformance, This would facilitate, for example, perhaps replacingquantized segments of the reference performance with corrected audiodata that was generated using the tempo map, providing an improvedtiming reference for learning, recording, and integrating an ensemble ofthe musical work.

FIGS. 20A-20F illustrate a guidance system used by musicians andadministrators, according to some embodiments. The system 2000 comparesthe frequencies performed, F_(P), by a musician to the frequenciestargeted by the musician, F_(T), within a select time-frame. Likewise,the system 2000 compares the timing of the frequencies performed,T_(FP), by a musician to the timing of the frequencies targeted by themusician, T_(FT), within a select time-frame 2010. The differencesbetween the frequencies performed and the frequencies targeted, and thedifferences between the timing of the performed frequencies and thetiming of the targeted frequencies, provide a performance measure, PM,for the musician and the administrator to follow in performing and/orreviewing the performance by the musician. The scale 2020 provides aqualitative display for use by the musician and/or the administratorused in the comparison. If the performance by the musician is “in thepocket” 2030, it can be considered to have differences in frequenciesand timing with the selected time-frame 2010 that are acceptable. Theperformance can be considered, qualitatively and/or quantitatively, ashaving the desired “groove” or “feel”, for example.

FIG. 20A illustrates an average of frequencies over the selectedtime-frame 2010 being slightly sharp in average frequency, and playedslightly slow, dragging. FIG. 20B illustrates an average of frequenciesover the selected time-frame 2010 being slightly flat in averagefrequency, and played slightly fast, rushing. FIG. 20C illustrates anaverage of frequencies over the selected time-frame 2010 being quiteflat in average frequency, and played slightly slow, dragging. FIG. 20Dillustrates an average of frequencies over the selected time-frame 2010being quite sharp in average frequency, and played slightly fast,rushing. FIG. 20E illustrates “the pocket”, and the musician playingquite a ways out of the pocket, not providing the desired performance ofgroove and/or feel. FIG. 20F illustrates the musician playing in thepocket, providing the desired performance of groove and/or feel. Theselected time-frame can be adjusted to provide a desired degree ofresolution during play, ranging from the representation of any beat ornumber of beats, measure or number of measures. In some embodiments, theselected time frame can include frequency and/or timing comparisons overthe entire performance. In some embodiments, the selected time-frame canrepresent frequency and/or timing comparisons in an interval rangingfrom 0.1 second to 1 minute, from 0.1 second to 50 seconds, from 0.1second to 40 seconds, from 0.1 second to 30 seconds, from 0.1 second to20 seconds, from 0.1 second to 10 seconds, from 0.1 second to 5 seconds,from 0.5 seconds to 30 seconds, or any range or 0.1 second therein. Insome embodiments, the selected time-frame can represent frequency and/ortiming comparisons in an interval ranging can represent 0.1 second, 1second, 2 seconds, 3 seconds, 5 seconds, 10 seconds, 15 seconds, 20seconds, 30 seconds, or any range therein in increments of 0.1 second.

FIG. 21 shows how a timeline guidance system can be used to help themusician stay “in the pocket” by monitoring note frequency and timingduring the course of a performance, according to some embodiments. Thesystem 2100 compares the frequencies performed, F_(P), by a musician tothe frequencies targeted by the musician, F_(T), within a selecttime-frame. Likewise, the system 2100 compares the timing of thefrequencies performed, T_(FP), by a musician to the timing of thefrequencies targeted by the musician, T_(FT), within a select time-frame2110. The differences between the frequencies performed and thefrequencies targeted, and the differences between the timing of theperformed frequencies and the timing of the targeted frequencies,provide a performance measure, PM, for the musician and theadministrator to follow in performing and/or reviewing the performanceby the musician. The scale 2120 provides a qualitative display for useby the musician and/or the administrator used in the comparison. If theperformance by the musician is “in the pocket” 2130, it can beconsidered to have differences in frequencies and timing with theselected time-frame 2110 that are acceptable. The performance can beconsidered, qualitatively and/or quantitatively, as having the desired“groove” or “feel”, for example.

The qualitative comparison can be made using the relative positions ofthe visual “cross-hairs” of the performed and targeted frequencies andtiming of the frequencies. A quantitative comparison can be made usingthe summation of the absolute values of the differences between theperformed and targeted frequencies and timing of the frequencies acrossthe selected time-frame. A mathematical relationship can be used, forexample:

(i) for a performance measure of the timing=PM_(T)=Σ|T_(FP)−T_(FT)| andΣ is taken for the timing of the performance of each frequency acrossthe selected time-frame; and,

(ii) for a performance measure of the frequency=PM_(F)=Σ|F_(P)−F_(T)|and Σ is taken for the performance of frequency across the selectedtime-frame.

These qualitative and quantitative measures can be used alone, or incombination with, a system of voting. The system of voting can beachieved through the votes of one or more administrators, such as one ormore directors, musicians in the plurality of musicians, users, or thegeneral public, for example. In fact, in some embodiments, the votingsystem alone can be used and can be implemented with or without thecomputer network.

Example 4. The Use of Audio Tempo Mapping to Index a Corresponding VideoPortion

This example illustrates steps that can be used in implementing theteachings herein to an audiovisual musical work. U.S. application Ser.No. 13/274,293 is hereby incorporated herein by reference in itsentirety.

Overdubbing to a Master Recording

This example shows how to use the methods provided herein combining theaudio track derived from a classic multi-track master recording, and theaudio/video performance of an instructor. Our example uses the originalmulti-track audio from Jimi Hendrix' “Purple Haze” and an audio/videoperformance of modern guitar legend Steve Vai demonstrating Jimi'stechnique.

The original multi-track master tapes were transferred to high qualitymulti-track audio files and mixed to sound indistinguishable from theoriginal version of the song. The original guitar performance and theaccompanying band track are both exported separately as stereo audiofiles so that the user will have discrete volume level control overthese elements. Using these isolated audio components, a timingreference is constructed as described herein. By utilizing theindividual instrument tracks from this original multi-track recording,the timing reference is created using a variety of notes, beats, andtransients.

Steve Vai enters the audio/video studio where his rendition of theclassic Jimi Hendrix song will be recorded with multiple camera anglesand high quality audio recording equipment. In addition to the bandtrack (all tracks remaining after removing the original guitar, forexample, using the teachings described herein and in U.S. Pat. No.7,902,446, which is hereby incorporated herein in its entirety byreference), Mr. Vai will use the timing reference as an accurate audioguide to perform his guitar in sync with the original band track. Eitherthe audio/video recording of Steve or the audio of the Hendrix song maybenefit from very subtle editing to match the 2 audio tracks in certainsections that will be very pleasing to the user.

Taking the finished and edited audio/video files of the performance andauthoring it in sync with the original Hendrix tracks, the user canchoose the desired camera angle for video playback and adjust the audioplayback levels of the band audio track, Steve Vai's performance audiotrack, the original Hendrix audio track and the metronome click track(directly derived from the timing reference). A user can now select adesired portion of the video to emulate, and the user's selection willautomatically move to the nearest cycle point and begin cycling throughthe selection. The cycling, for example, can be further defined by adefault cycle point selection which might be 2 beats, 1 bar, 2 bars oran entire section such as a verse, chorus, or the entire song. Thetiming reference provides the point cycle feature with the informationit needs so the user can quickly select a desired section of theaudio/video and watch the same performance over and over again, and cando so with the synchronized audio cycling. In this way, the user canvery quickly learn every technique that Steve Vai demonstrated on theHendrix song. The user can select a small 1 bar section with the pointcycle feature turned on in advance and as the section is perfected, theuser can select the next closest start point or end point allowing theuser the ability to change his start and end selection quickly andproficiently to the next closest cycle point at the beginning or endingof the current selection. The user will benefit from the ability to slowdown the speed of the audio/video performance as desired withoutchanging the pitch of the audio and examine the detail and nuances ofSteve Vai performing his rendition of Purple Haze. As such, this allowsthe user to emulate this performance in slow motion with a great degreeof detail and confidence and, once mastered, the user may now be able toemulate the performance at regular speed.

Example 5. Uploading and Displaying Multiple Video Portions as an“Ensemble” of Performances that are Synchronized Through a TimingInterface

This example illustrates steps that can be used to receive, assemble,and display multiple audiovisual data files from a plurality ofmusicians that were performed uploaded at locations remote from thereceiving, assembling, and displaying.

The methodology includes receiving a plurality of submissions from theplurality of musicians and using one variable tempo map created for theperformance of each submission to transform data in each submission intoa chronological series chunks of bars or beats that correspond to thetempo map, such that each audiovisual file is triggered to play insynchronization with the other submissions. The chunking of dataprovides one or more triggering-events for aligning and playing each ofthe plurality of submissions together as an ensemble.

As described herein, musicians are provided with a system forcontributing to an ensemble of a set of remotely-performed andremotely-uploaded musical performances through a computer network. Thesystems have a processor operably connected to a memory and an inputdevice operable to receive audio data. The memory can include a databaseoperable to store data for access; a timing reference having (i)instructions for execution by the processor and (ii) a tempo selected bya director, for example; an audio engine embodied in a non-transitorycomputer readable storage medium, wherein the audio engine is operableto transform input audio data to output audio data; and, an instructionmodule operable for providing directions for a musician. The instructionmodule can direct the musician to record a musical submission at alocation that is remote from a director of the ensemble, the recordingincluding configuring the musical submission by following the timingreference when performing the musical submission. The instruction modulecan also direct the musician to upload the musical submission to thecomputer network at the location remote from the director to enable thedirector to create the ensemble by integrating the musical submissionwith the second musical submission using the timing interface.

As such, the timing interface is integral to aligning the musicalsubmissions as an ensemble. A technical hurdle remains, in that theotherwise alignable musical submissions now need a triggering event toinstruct each of the submissions to begin playing at the same timethrough a computer network. This is a significant technical hurdle, asthe audiovisual files will typically include two batches of data—onevideo batch and one audio batch, the audio batch having, for example,two separate mono files or one interleaved stereo file. One of skillwill appreciate that, although each submission contains files that weremeant to play together, synchronized, starting and stopping at generallythe same time, such that they play in unison, the art does not designthe separate submissions to play together, synchronized, starting andstopping at generally the same time, such that they play in unison.

The process for creating the triggering event can include, for example,receiving each separate submission as a combination of audio and videodata. For example, an ensemble created through the selection of 5submissions will include 5 batches of video data and 5 batches of audiodata, each batch of data performed using the timing reference, and eachneeding a triggering event through the computer system for playbackthrough a common, master computer clock.

The common triggering event can be obtained, for example, by combiningthe audio data batch and video data batch from each submission into acommon AV_(i) file. An ensemble of 5 submissions, for example, willinclude common files designated as AV₁, AV₂, AV₃, AV₄, and AV₅. Each ofthe submissions will contain the common timing reference to serve as thetiming interface, and the triggering event can occur as follows:

-   -   1. Align the timing reference of AV₁ with a SMPTE time code, the        timing reference based on a tempo map that quantizes the        performance into beats and bars. The SMPTE time code can be used        to trigger the start of the playback of AV₁.    -   2. Align the timing reference of AV₂ with a SMPTE time code, the        timing reference based on a tempo map that quantizes the        performance into beats and bars. The SMPTE time code can be used        to trigger the start of the playback of AV₂ with AV₁.    -   3. Align the timing reference of AV₃ with a SMPTE time code, the        timing reference based on a tempo map that quantizes the        performance into beats and bars. The SMPTE time code can be used        to trigger the start of the playback of AV₃ with AV₁ and AV₂.    -   4. Align the timing reference of AV₄ with a SMPTE time code, the        timing reference based on a tempo map that quantizes the        performance into beats and bars. The SMPTE time code can be used        to trigger the start of the playback of AV₄ with AV₁, AV₂, and        AV₃.    -   5. Align the timing reference of AV₅ with a SMPTE time code, the        timing reference based on a tempo map that quantizes the        performance into beats and bars. The SMPTE time code can be used        to trigger the start of the playback of AV₅ with AV₁, AV₂, AV₃,        and AV₄.    -   6. Optionally, the files can remain as separate audio and video        files A₁V₁, A₂V₂, A₃V₃, A₄V₄, and A₅V₅.

Having the timing reference and SMPTE map overlay allows for a dynamicpoint referencing of the playback of the ensemble, the playbacksynchronization based on this method controlled by a common master clockfrom which the ensemble is combined and played. This technology providesa powerful new tool for uploading a plurality, perhaps massive, numberof submissions, reviewing the submissions individually and in anycombination, while having the ability to select any section or sectionswithin a musical work for comparison, review, looping, selection, andswapping out of musicians to form an ensemble.

The system can also include a correction or “re-queue” feature, in whicha frame or plurality of frames can be added or removed to shift the timecode of a data stream. For example, the SMPTE time code can be used toinitiate a triggering event. This triggering event can be followed by aseries of re-triggering events, the frequency of re-triggering beingselected as desired. The frequency of the re-triggering event can dependon the tempo and “looseness” of a particular song. A song having a tempoof 120 bpm may be appropriately re-triggered every 4 measures, or 16beats, in some embodiments. A range of re-triggering events may occur,in many cases, at every 1 beat to 8 bars, in some embodiments.

The current state-of-the-art does not offer a technology for assemblingmultiple performances as an ensemble. In some embodiments, the term“multiple” can be used to refer to 3, 4, 5, 6, 7, 8, 10, 12, 14, 16, 18,20, 30 or more performances that can be separately performed, separatelyuploaded, and assembled as an ensemble.

It should be appreciated that, if the processing requirement exceeds amachines power, it may be desirable to add additional processingcapability. This can be in the form of a separate processing core foreach video stream, in some embodiments. An external master clock sourcemay also be desired to ensure a reliable synch between machines. A blackburst generator, for example, can reference the common power frequencyof 60 cycles/second with the master clock pulling down the speed of allsynchronized devices from 30 frames per second to a black burst resolvedspeed of to 29.97 frames per second. This ensures that all devices arerunning at the same resolved speed.

Example 6. Use of the Systems and Methods in the Audition Process

The methods and systems provided herein can be used in the auditionprocess.

Auditioning a new singer or musician for a musical group can be a timeconsuming task. Musicians aren't known for their punctuality, forexample, making it difficult to have back-to-back auditions without longand costly breaks waiting for the next participant. Moreover, theenvironment of a typical audition is not expected to be comfortable,often hot and in a low-cost overhead urban environment, making theoutside and the inside of the audition location generally unappealing.As such, one of skill in the art of auditions will appreciate thatwatching the auditions at your own pace and in your own environmentwould be preferred.

In addition, the audition environment is not conducive to the creationof a performer's best work. For example, a nervous singer showing up ina room packed with people that they've never met is an experience thatmakes it challenging at best to give the best performance they have togive. The systems and methods provided herein allow auditions to beviewed off-line at a later date, reducing the stress significantly, asthere is no need for 25 or so people to be staring at the singer in asmall, undesirable rehearsal space with florescent lighting.

As such, one of skill in the art of auditioning will appreciate thesystems and methods provided herein.

Example 6. Use of the Systems and Methods in the Process of Practicing aPerformance

The methods and systems provided herein can be used in the process ofpracticing a performance.

All forms of artistic expression require practice to refine inpreparation for a performance. Many Church bands, for example, learn newcontent every week and the leader of the band can find him or herself ina constant state of frustration. For example, a person that didn't learntheir part correctly can make the rest of the band very irritable due tocreating the need for everyone to spend more hours at practice thananticipated, cutting into their social or family time.

The leader can now stay home at least one night a week, choosing to waitand confirm that all participants have learned their parts correctly.

As such, one of skill in the art of practicing a performance of anytype, particularly a group performance, will appreciate the systems andmethods provided herein.

Example 7. Use of the Systems and Methods in the Process of Assembling aSet of Performances

This example provides a concise illustration of a method and system forobtaining, selecting, and assembling a set of video performance for aconcurrent, synchronized playback of otherwise remote performances.

FIG. 22 is and illustration of how video selections of multipleperformances can be selected and placed on a graphical user interfacefor concurrent viewing, according to some embodiments. Each instrumentperformance submitted, for example, is buffered and sent to a streamingvideo playback module 2240. First, the control module 2205 manages thevideo selection criteria for each of a set of user video files fordisplay. In FIG. 22, 5 different user video files, each playing adifferent instruments is presented on one display, or a plurality ofdisplays.

The control module 2205 regulates the selection of the user video filesfrom instrument bank 1 2210, instrument bank 2 2215, instrument bank 32220, instrument bank 4 2225, and instrument bank 5 2230. The instrumentbanks 2210,2215,2220,2225,2230 are categorized, respectively, torepresent each of 5 instruments that are eligible to be included in asynchronized streaming playback of 5 select videos. The videos arestreamed from the instrument banks through the synchronizer/video buffer2235 and into the streaming video playback module 2240. The controlmodule, the instrument banks, the synchronizer/video buffer, and thestreaming video playback module are operably connected to a processorfor executing the instructions for playback of the videos on a graphicaluser interface.

The instrument banks 2210,2215,2220,2225,2230 provide a database ofpotential videos for selection, and one video is selected for playbackper instrument bank based on dynamic criteria that may include, but isnot limited to, user voting, electronic performance evaluation, adminoverride and/or random selection. The synchronizer/video buffer module2235 aligns the timing of the playback of the selected video files forsynchronous streaming 2240 using the methods and processes taughtherein.

In some embodiments, the display can be assembled using the streamingvideo playback module 2240. The module can execute instructions on theprocessor to assemble each of the 5 videos (videos 1 through 5) bylocating each of the videos on a single screen space in its owngeometrical area by positioning each video in its own unique space,within the borders of a single screen. Each of the 5 unique spaces couldbe exactly the same geometrical area or radically different. Forexample, 4 of the geometrical areas can be defined as relatively smallerfor the musicians (each less than 3%-20% of the entire single screen,for example) as compared to a larger, perhaps huge, area for the singer(or 80%-97% of the entire screen).

In some embodiments, each user video file in each instrument bank willbe cut into predefined smaller files. In some embodiments, thepredefined smaller files will correspond in time to one or more bars. Insome embodiments, the predefined smaller files will correspond in timeto one or more beats. For example, the predefined smaller files cancorrespond to about the length of 1 bar of music at 120 bpm. In anotherexample, the predefined smaller files can correspond to a length ofabout 1 msec, 2 msec, 3 msec, 5 msec, 10 msec, 50 msec, 100 msec, and upto 1 minute in increments of 1 msec. Each of these predefined smallerfiles can be triggered approximately every bar or by a predeterminedtrigger point based on realtime, time code, bar, beats etc.

In some embodiments, the user video file in each instrument bank can becut into predefined smaller files based on a variable tempo mapping ofthe performance and making a start point at the beginning of each barand a stop point at the end of each bar. These start and stop pointscould also be adjusted in increments ranging in duration from about 1millisecond to about 100 milliseconds, from about 5 milliseconds toabout 50 milliseconds, from about 10 milliseconds to about 100milliseconds, from about 20 milliseconds to about 100 milliseconds, fromabout 2 milliseconds to about 20 milliseconds, from about 3 millisecondsto about 30 milliseconds, from about 4 milliseconds to about 100milliseconds, or any range or amount therein in increments of 1millisecond.

In addition, the user video file in each instrument bank can be cut intopredefined smaller files based on a variable tempo map or tempo defined“sniffer,” making a start point at the beginning of each bar or bars anda stop point at the end of each bar or bars. These start and stop pointscould also be individually or globally adjusted in increments ranging induration from about 1 millisecond to about 100 milliseconds, from about5 milliseconds to about 50 milliseconds, from about 10 milliseconds toabout 100 milliseconds, from about 20 milliseconds to about 100milliseconds, from about 2 milliseconds to about 20 milliseconds, fromabout 3 milliseconds to about 30 milliseconds, from about 4 millisecondsto about 100 milliseconds, or any range or amount therein in incrementsof 1 millisecond.

After the cuts (edits) have been made, for example, each of thesepredefined smaller files can be triggered approximately every bar or bya predetermined trigger point based on realtime, time code, bar, beatsetc.

We claim:
 1. A system for a real-time integration and review of anensemble of a set of remotely-performed and remotely-uploaded danceperformances through a computer network, the system comprising: aprocessor operably connected to a memory on a non-transitory computerreadable medium; a receiving module operable for receiving video datafrom a dance submission and a second dance submission on anon-transitory computer readable medium; an integration engine embodiedin a non-transitory computer readable medium, wherein the integrationengine is operable to interface the timing of a dance submission withthe timing a second dance submission, in real time, using a timingreference track that includes a tempo map in the creation of anensemble; an instruction module on a non-transitory computer readablemedium operable for providing real-time directions for each of theplurality of dancers; and, a display module operable for displaying theensemble; wherein the timing reference track is selected from the groupconsisting of: a click-track; a drum track; a select reference track ofa select reference performance, , the select reference performancehaving a series of sound frequencies, each of the frequencies performedat a particular time in the select reference performance; a residualaudio track representing a multi-track digital audio file, the residualtrack having a series of sound frequencies, each of the frequenciesperformed at a particular time in the residual audio track; and, acombination thereof; wherein, the system enables a real-time integrationand review of the performances to assess one performance relative toanother performance in real-time.
 2. The system of claim 1, wherein thereal-time review is provided by a real-time instructor.
 3. The system ofclaim 1, further comprising an expert analysis module providing aperformance rating in real-time by a professional live teacher.
 4. Thesystem of claim 1, wherein the timing reference used by the integrationengine is one track in each of a series of multi-track digital audiofiles available upon request by each of a plurality of dancers; each ofthe series containing a shared musical work, highlighting a selectreference performance on a select reference track within the musicalwork, and including (i) T reference performances within the musicalwork, wherein T=Σ

_(i); and,

_(i)=

₁,

₂,

₃, . . . , or

_(T); (ii) a select reference track of the select reference performance,

_(i), the select reference performance having a series of soundfrequencies, each of the frequencies performed at a particular time inthe select reference performance; (iii) a residual audio trackrepresenting the multi-track digital audio file having the selectreference performance removed; and, (iv) instructions for execution ofthe timing reference by the processor.
 5. The system of claim 1, furthercomprising an input device operable to receive audio data on anon-transitory computer readable medium; an audio engine embodied in anon-transitory computer readable medium, wherein the audio engine isoperable to transform input audio data to output audio data as an audiorecording of a dance submission for display in a visual ensemble; anoutput module embodied in a non-transitory computer readable medium,wherein the output module is operable to transmit the integrated audiodata, along with the video data, to an output device, the output deviceoperable to display the visual ensemble.
 6. The system of claim 1,further comprising a guidance module on a non-transitory computerreadable medium and operable as a guidance system including a timingcomparison tool to compare the dance submission to the select referenceperformance, the guidance system providing a real-time feedback thatincludes a qualitative or quantitative measure of the timing comparison.7. The system of claim 1, wherein the a review of the performances toassess one performance relative to another performance is a real-timeperformance mapping of the one performance against the otherperformance.
 8. The system of claim 1, further comprising a guidancemodule on a non-transitory computer readable medium and operable as aguidance system including a frequency comparison tool and a timingcomparison tool to compare the dance submission to the select referenceperformance, the guidance system providing a real-time feedback thatincludes a qualitative or quantitative measure of the frequencycomparison and the timing comparison.
 9. The system of claim 1, furthercomprising a dynamic guidance module on a non-transitory computerreadable medium and operable as a dynamic guidance system including afrequency comparison tool and a timing comparison tool to provide areal-time continuous comparison of the dance submission with the selectreference performance from the beginning of the performance to the endof the performance, or any section therein, the dynamic guidance systemproviding a real-time qualitative or quantitative measure of thefrequency comparison and the timing comparison.
 10. The system of claim1, further comprising a data exchange module embodied on anon-transitory computer readable medium, wherein the data exchangemodule is operable to exchange data with external computer readablemedia for real-time communications.
 11. A method of facilitating thereal-time integration and review of an ensemble of a set ofremotely-performed and remotely-uploaded dance performances through acomputer network, the method comprising: engaging in a computer networkhaving a processor operably connected to a memory on a non-transitorycomputer readable medium; a plurality of dancers, each of the pluralityof dancers (i) at a remote location from an administrator that selects aset of performances, Σ

_(ir)≤

_(1r),

_(2r),

_(3r), . . . , +

_(Tr), from the plurality of dancers in creating an ensemble and (ii) inan operable communication with the computer network; a series ofmulti-track digital audio files available upon request by each of theplurality of dancers; each of the series containing a shared musicalwork, highlighting a select reference performance on a select referencetrack within the musical work, and including (i) T referenceperformances within the musical work, wherein T=Σ

_(i); and,

_(i)=

₁,

₂,

₃, . . . , or

_(T); (ii) a select reference track of the select reference performance,

_(i), the select reference performance having a series of soundfrequencies, each of the frequencies performed at a particular time inthe select reference performance; (iii) a residual audio trackrepresenting the multi-track digital audio file having the selectreference performance removed; (iv) instructions for execution by theprocessor; and, directions instructing to each of the plurality ofdancers to request one of the series of multi-track digital audio fileshaving the select reference performance; and, perform their submissionusing the select reference track, the residual audio track, or thecombination thereof, as a timing interface for integration of theensemble; wherein, the engaging includes integrating and reviewing theperformances to assess one performance relative to another performancein real-time.
 12. The method of claim 11, wherein the engaging includesdirecting, and the method further comprises: reviewing the plurality ofsubmissions in real-time; and, interfacing the set of performances withan integration engine on a computer readable medium to create theensemble for displaying with a display module on a non-transitorycomputer readable medium in real-time.
 13. The method of claim 11,further comprising offering a guidance system including a timingcomparison tool to compare the dance submission the select referenceperformance, the guidance system providing a real-time feedback thatincludes a qualitative or quantitative measure of the timing comparison.14. A method for a dancer to contribute a real-time performance to anensemble of dance performances from a location that is remote from thelocation of an administrator that is creating the ensemble over acomputer network, the method comprising: entering a computer networkhaving a processor operably connected to a memory on a non-transitorycomputer readable medium; a plurality of dancers, each of the pluralityof dancers (i) at a remote location from an administrator that selects aset of performances, Σ

_(ir)≤

_(1r),

_(2r),

_(3r), . . . , +

_(Tr), from the plurality of dancers in creating an ensemble; and (ii)in an operable communication with the computer network; a series ofmulti-track digital audio files available upon request by each of theplurality of dancers; each of the series containing a shared musicalwork, highlighting a select reference performance on a select referencetrack within the musical work, and including (i) T referenceperformances within the musical work, wherein T=Σ

_(i); and,

_(i)=

₁,

₂,

₃, . . . , or

_(T); (ii) a select reference track of the select reference performance,

_(i), the select reference performance having a series of soundfrequencies, each of the frequencies performed at a particular time inthe select reference performance; (iii) a residual audio trackrepresenting the multi-track digital audio file having the selectreference performance removed; (iv) instructions for execution by theprocessor; and, directions instructing to each of the plurality ofdancers to request one of the series of multi-track digital audio fileshaving the select reference performance; perform their real-time dancesubmission at their remote location using the select reference track,the residual audio track, or the combination thereof, as a timinginterface for streaming and integration of the ensemble; and, requestingfrom the computer network one of the series of multi-track digital audiofiles having the select reference performance that is to be replacedwith a submission,

_(ir); performing the streaming at a location remote from theadministrator while using the select reference performance as a guidefor the timing of performing each of the frequencies, and using theresidual audio track as an accompaniment in a streaming of thesubmission for an interfacing of the submission with a second submissionby a second dancer at a second location remote from the administratorwhile using the select reference performance as the timing interface,and using the residual audio track as an accompaniment; and, adisplaying of the submission with the second submission as an ensemble.15. The method of claim 14, wherein the ensemble is a visual ensemble,and each performance in the set of performances includes a real-timevideo streaming of the respective submission for display in the visualensemble.
 16. The method of claim 14, further comprising using aguidance system including a timing comparison tool to compare the dancesubmission to the select reference performance, the real-time guidancesystem providing a real-time feedback that includes a qualitative orquantitative measure of the timing comparison.
 17. The method of claim14, wherein the administrator is a director that is auditioning fortalent.
 18. The method of claim 14, wherein the administrator is asponsor of an audition.
 19. The method of claim 14, wherein theadministrator is a dance instructor.
 20. The method of claim 14, whereinthe administrator is a professional performing artist.